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Research Highlights 


Rationale: 

The newly designed Illinois High School to College 
Success Report is one of the first collaborative 
endeavors among the Illinois Board of Higher 
Education, the Illinois Community College 
Board, and the Illinois State Board of Education 
to combine their fairly disparate data systems. It 
fulfills the requirement of Illinois Senate Joint 
Resolution No. 59 (October 2007) to develop a 
new report to further inform schools and districts 
about their graduates’ postsecondary outcomes. As 
Illinois continues to develop its longitudinal data 
system using a federated data model, it is critical 
to evaluate the dissemination and usefulness of the 
High School to College Success Report, as well as 
solicit suggestions for improvement, in order to 
guide development of future reports. 

Purpose: 

The evaluation study examined the following areas: 

1) perceptions of the dissemination of the report; 

2) how the HS2CSR is being utilized and how useful 
state educators perceive the report; 3) how users 
are engaging in collaborative efforts utilizing the 
HS2CSR and in other college readiness activities; 
and, 4) suggestions for improving the report. 

Methods: 

The evaluation included a mixed-method approach 
with an online survey, focus groups (face-to-face, 
online, telephone conference call), and focused 
conversations at three Illinois education meetings. 
All data were collected in Spring 2013. 

• Online survey garnered usable responses 
from 373 (23% response rate) from high 
school principals, superintendents, regional 
superintendents, 2-year and 4-year senior 
administrators, and individuals from high 
schools, community colleges, and 4-year 
institutions who participated in workshops 
hosted by IBHE, ICCB, and ISBE. 


• Detailed, rich qualitative data were gathered 
from eight focus groups with a total of 20 
individuals including high school principals, 
superintendents, regional superintendents, 
and 2-year administrators. 

• Qualitative feedback was obtained during two 
Illinois education meetings with postsecondary 
senior administrators (e.g., 2-year and 4-year 
Chief Academic Officers) and a meeting with 
the IBHE Faculty Advisory Council. 

• The survey data were analyzed with descriptive 
statistics and the qualitative data were analyzed 
for overall themes. 

Findings: 

Results from the survey analysis and qualitative 
data affirmed the usefulness of the HS2CSR to 
facilitate improved student achievement by fostering 
communication and collaboration between K-12 
and postsecondary educators and policymakers. 
The HS2CRS was typically used to monitor student 
success. More collaborative activity utilizing the 
report occurred within the K-12 sector and occurred 
within institutions, rather than cross-institutions. 
Results also identified limited report dissemination 
and concerns about the comprehensiveness of the 
data. Suggestions from study participants to improve 
the report included increasing the ease of use, 
increasing the representativeness of the data, and 
providing additional data elements. 

Recommendations: 

• Improve dissemination by utilizing email 
notification, targeting specific subgroups 
invested in successful secondary to postsecondary 
transitions (e.g., regional superintendents, 
high school and postsecondary counselors, 
institutional researchers, and secondary and 
postsecondary faculty advisory councils) and 
using multiple distribution methods (e.g., 
IBHE memo, ISBE superintendent weekly 
message, Regional Offices of Education). 
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• Increase the use of the report by 
enhancing its value by improving 
the representativeness of the data 
(e.g., private institutions, out-of-state 
institutions), relying less on self-report 
data, and including additional data 
elements (e.g., trend data, comparison 
data) . 

• Increase ease of use of the report by 
providing an executive summary, 
adding data definitions and data sources, 
supporting users’ data technical needs 
through training and data coaches, and 
offering an online relational database 
for users to access the data and create 
custom reports. 

• Increase collaboration within and across 
institutions by convening workshops 
and webinars on interpretation and 
use of the report and engaging current 
high-end users to share their strategies 
for collaboration and experiences 
utilizing the report, particularly around 
alignment of the new Common Core 
State Standards. 
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Introduction 


The newly designed Illinois High School to College Success Report (HS2CSR) has been 
in the making for the past 20 years. It was formerly known as the Illinois High School 
Feedback Report, produced by the community colleges and provided to their in-district 
high schools. The HS2CSR fulfills the requirement of IL Senate Joint Resolution No. 
59 (October 2007) to develop a new report to further inform schools and districts 
about their graduates’ postsecondary outcomes. The major purpose of the report, as 
described on the Illinois Board of High Education’s website (http://www.ibhe.state.il.us/ 
HSCSR) is “to facilitate improved student achievement by fostering communication 
and collaboration between K-12 and postsecondary educators and policymakers.” 

The newly designed Illinois High School to College Success Report is one of the first 
collaborative endeavors among three state education agencies, Illinois State Board of 
Education (ISBE), Illinois Community College Board (ICCB), and Illinois Board of 
Higher Education (IBHE), to merge their disparate data systems. The state agencies 
collaborated with American College Testing Services (ACT) to produce the feedback 
report because Illinois is a universal ACT state (i.e., all public school juniors take the 
ACT) and ACT had developed a report template which could be modified to meet the 
educational institutions’ needs. As a precursor to the development of the new Illinois 
Longitudinal Data System, the partnership among these three state education agencies, 
in collaboration with ACT, produced the HS2CSR to provide Illinois public high schools 
with information about their graduates’ postsecondary outcomes and to provide Illinois 
public community colleges and universities with college-readiness information about 
their incoming freshmen students. 

According to a recent report by the Data Quality Campaign (DQC, 20 1 3) ’, Illinois is one 
of 47 states that provides a high school feedback report. The DQC report summarized 
their analyses of which states produce a feedback report and how many meet their criteria 
(i.e., transparent, actionable, and timely) for an effective report. They also provide a 
state-by-state analysis of each of the states’ high school feedback reports. The DQC 
analysis found that of the 47 states that produce a high school feedback report, 38 states 
make the reports publically available (transparent), 29 produce the report by high school 
(actionable), and 24 include recent data (i.e., since 2010, timely). Their analyses rated 
Illinois meeting the two criteria of transparent and timely; however, the Illinois HS2CSR 
was not deemed actionable. The DQC summary of Illinois information indicates that the 
HS2CSR is not available to individual high schools; however, an ICCB representative (B. 
Durham, personal communication, July 9, 2013) confirms that, in fact, the HS2CSR 
does provide individual high school reports. As can be seen in the DQC state-by-state 
analysis, there are many different types of high school feedback reports. According to our 
ACT contact (G. Schlott, personal communication, July 2, 2013), Illinois is one of five 
states (i.e., Illinois, Arkansas, Louisiana, West Virginia, and Wisconsin) that currently 
uses the ACT template to produce their “High School to College Success” report. We 


Illinois is one of 47 
states that provides a 
high school feedback 
report. Illinois meets 
DQC criteria for 
an effective report: 
transparent, actionable, 
timely. 


1 http://www.dataqualitycampaign.org/files/l662_HS%20Feedback%20State%20Analysis.pdf 
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spoke with representatives from Wisconsin, as well as Tennessee (TN previously used 
ACT template), to learn if they monitored use or feedback about their reports (see 
sidebar story). In short, Wisconsin has recently begun to monitor who views the online 
report; however, they do not keep records on utilization of the report. For the Tennessee 
report, school administrators use the report to track postsecondary outcomes for their 
graduates; however, the state does not conduct any formal research to track who uses 
the report. Neither state reported collecting feedback information about their reports, 
although Tennessee has convened a committee to research specific needs for their new 
state-produced report currently under development. 

On June 24, 20 1 1 , IBHE, ICCB, and ISBE, along with ACT hosted a statewide summit 
in Springfield, Illinois, to announce the release of the new Illinois Eligh School to College 
Success reports with over 200 people in attendance. The first annual EIS2CS reports were 


Conversations with Wisconsin and Tennessee 


We spoke with state education representatives from 
Wisconsin and Tennessee in May 2012 about their High 
School to College Success Report to learn how their 
states utilize the report and if they monitor the use of 
the report. The Senior I nstitutional Planner in the Office 
of Policy Analysis and Research at the University of 
Wisconsin indicated Wisconsin has used the ACT report 
since the mid-1990s with a new form of the report 
developed every three years. Compared to Illinois, 
both Wisconsin and Tennessee have a wider audience 
for primary distribution, which also includes admission 
officers, institutional research directors, guidance 
counselors, and chief instructional officers. Starting in 
2010 Wisconsin distributed the report through a secure 
website, which allowed for monitoring of schools that 
actually accessed the report. They found that in 2010 
roughly one-third of the reports were viewed by the 
schools. Although Wisconsin has begun monitoring 
who views the report, they do not keep records of 
how the report is used. Wisconsin has also not been 
involved in any analysis or research of how the report 
is used nor do they offer any assistance to schools to 
help them use the report. 

According to the Chief Policy Officer at the Tennessee 
Higher Education Commission, Tennessee has used 
the report for the past five years; however, they made 
plans to develop their own report with the creation of 
their new P-20 data system over the past couple of 
years. Tennessee distributed their report by email 
with a Listserv or in hard copy. Several years ago, 
members of their higher education department traveled 
with ACT to provide a training seminar on the data 
and potential uses of the report to over 800 state 
guidance counselors. The training only occurred at 
that time and no further training has been provided. 
Due to state webpage requirements, access to the 
report is very difficult as it is layered within several 


links on the higher education department's webpage, 
potentially resulting in less access than would be 
available with an easily accessible link. One positive 
aspect is that all of the reports, regardless of education 
sector, are available through this one website and are 
easily accessed through drop-down menus. The Chief 
Policy Officer believed that currently the media uses 
the report to make comparisons between schools. He 
also said high school administrators use the report 
to identify which universities the schools' graduates 
attend and to identify ways to improve the high 
school curriculum. He believes the report needs to 
be improved to examine retention over a span of four 
years and examine students' performance in math and 
English classes at the high school and postsecondary 
levels. Tennessee hopes to increase the usefulness 
and accessibility of the new report. I n the future after 
a new report is created, Tennessee hopes to provide 
grant opportunities for universities and high schools to 
receive training on using and implementing the report’s 
results to help improve students' transition from high 
school to college. Additionally, Tennessee currently does 
not monitor use or feedback on the report; however, it 
has created a committee to research the specific needs 
for the state in order to design the best report possible. 

I n a follow-up phone call in J uly 2013, the Chief Policy 
Officer indicated their new high school feedback report 
will be rolled out in Fall 2014. With the development of 
their new P-20 data system, the new report will be able 
to include actual behavior data (e.g., course taking) 
rather than self-report data, as well as additional years 
of data beyond the first college going year. I n addition, 
Tennessee will utilize Tableau interactive data software 
to increase flexibility in using the data. He believes 
these improvements, along with the report coming from 
their high profile new P-20 data system will increase 
the buy-in and use of their new report. 
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released and disseminated on the same day to public high schools, community colleges, 
and universities, as well as made available to the public on the agencies’ websites. These 
reports described performance indicators for ACT-tested, first-time, full-time students 
of 2006-2008 who attended public postsecondary institutions in the falls of 2006-2008. 
A second report was released in February 2013 for ACT-tested, first-time, full-time 
students of 2008-2010 who attended public postsecondary institutions in the falls of 
2008-2010. Information that institutions receive from the reports includes the following: 

• Students’ performance (GPA) at a 2- or 4-year postsecondary institution 

• Students performance compared to state average 

• College readiness ACT Benchmark Scores by subject area 

• Aggregate data on students’ performance in college, as well as high school 

• Data disaggregated by school/ college 

• Student performance by high school course sequence patterns in math and science 

• Average Fall CPAs by ACT College Readiness Standards score ranges 

• Student performance in credit bearing and developmental courses 

• Persistence from year one to year two of postsecondary career, as well as persistence 
at the same institution 

The desired outcomes of these reports were: 

• To facilitate collaborative discussions between secondary and postsecondary 
institutions using common data points 

• To encourage data-driven decision making 

• To support secondary and postsecondary institutions in developing an articulated 
and aligned curriculum 

• To help in the evolution toward a robust P-20 and effective longitudinal data 
system. 

The overall goal of the HS2CSR was to increase the usefulness of data presented in the 
report with the intent of transitioning to a state-developed report. As Illinois builds 
its longitudinal data system and the capacity to draw upon enhanced data metrics, 
the ultimate goal was that the state will design and produce standardized reports for 
secondary and postsecondary institutions, rather than depending on an external agent. 

As part of the development process of the HS2CSR, state education agencies planned 
to conduct an evaluation of the effectiveness and use of the report, once the new report 
was disseminated and institutions became familiar with using the HS2CSR. This 
report summarizes the evaluation study and findings. The following sections describe 
the evaluation methodology and study results and provide recommendations for the 
development of future reports. 
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Methodology 


The evaluation study sought to better understand perceptions of the delivery and 
dissemination of the report, how report users are engaging in collaborative efforts, and 
suggested data elements to add to, or remove from, the report. The study used a multi- 
method approach to gather feedback from a wide variety of education stakeholders: 
principals, superintendents, Regional Offices of Education superintendents, 2-year and 
4-year senior administrators, and various individuals from the high school, community 
college, and 4-year institution sectors. The evaluation methods included an online survey 
and focus groups (face-to-face, online, telephone conference call). (See Appendix A 
and B for a copy of the survey and focus group protocol). In addition, feedback was 
obtained from individuals at three Illinois education meetings with postsecondary senior 
administrators and faculty. 

Individuals (N= 1,651) from Illinois public K-12 and postsecondary institutions were 
invited to participate in an online survey to provide feedback on the dissemination 
and usefulness of the HS2CSR. They also provided information on their collaborative 
activities and suggestions for improving the report. An email pre-notice was sent 
informing individuals about the purpose of the study and the upcoming online survey 
invitation. The invitation was sent one week after the distribution of the Year 2 HS2CSR 
in February 2013. Four email reminders were sent approximately one week apart, 
ending survey data collection at the end of March 2013. Notices about the survey 
were posted on the IBHE Friday Memo, the ISBE Superintendent’s weekly message, 
and the Illinois Principal’s Association electronic newsletter. We surveyed all K-12 
high school principals (n=6 00) and district superin tendents/regional superintendents 
(n= 523) from Illinois public schools and senior administrators from 2-year (n= 113) 
and 4-year (n= 93) Illinois public institutions from the 2012-2013 academic year. The 
senior administrators included presidents/chancellors, chief academic officers/ provost, 
and directors of enrollment. In addition, 322 participants who attended professional 
development workshops sponsored by IBHE, ISBE, and ICCB on the new report were 
surveyed. Workshop participants typically included teachers and faculty from K-12 
districts and community colleges. Contact names and email addresses were obtained 
from publicly available websites. Contact information for workshop participants was 
provided by a state education agency. Some individuals forwarded the survey invitation 
to one of their staff to complete and return. Survey results were analyzed with descriptive 
statistics using frequencies and cross-tabulations. 

K-12 public high school principals, district superintendents, regional superintendents, 
and 2-year community college administrators were given the option to provide more in- 
depth feedback through focus groups. An invitation to participate was emailed to those 
on the initial survey list. Potential participants were asked if they would be interested in 
participating in a focus group for the purpose of providing in-depth feedback and if so 
what medium would they prefer (online, telephone, or face-to-face) . The individuals who 
indicated interest in participating in a focus group were contacted through email to set 
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up times for their participation. These participants self-identified, which increased the 
likelihood of selecting individuals with experience using the report or increased interest 
in using the report compared to selecting a random sample of individuals. 

The focus groups were conducted from May to June 2013 and through three different 
mediums: online focus groups through Go-to-Meeting (an online meeting software), 
face-to-face focus groups, and telephone conference calls (with 2-4 participants). It is 
important to note that in order to better accommodate the participants, locations for 
face-to-face focus groups were chosen based on the locations of the participants. A few 
central locations were chosen with the aim at preventing participant travel from exceeding 
60 miles one way. We attempted to provide as many opportunities to participate as 
possible. There were a total of 8 focus groups conducted, 3 face-to-face, 4 telephone, 
and 1 online, in which a total of 20 individuals participated. All focus groups were audio 
recorded. In addition to the focus groups, researchers attended three separate meetings 
with postsecondary senior administrators and faculty to request feedback on the focus 
group questions. The meetings were held in March and May 2013 and included: 1 ) Chief 
Academic Officers (CAOs) of Illinois community colleges held at Richland Community 
College, 2) CAOs of 4-year and community colleges held at IBHE, and 3) IBHE Faculty 
Advisory Council (FAC) which includes faculty from both 2-year and 4-year institutions 
held at Monmouth College. The meetings with the CAOs were audio recorded and notes 
were collected at the IBFIE FAC meetings. Thus, the qualitative data analysis included a 
total of 10 recordings plus notes taken at the IBHE FAC meeting. The recordings were 
transcribed and reviewed independently to develop a coding scheme that served as a 
conceptual framework by which to organize the data. All of the transcripts were coded 
by one research team and reviewed for inconsistencies by another research team, with 
oversight from a senior researcher. Consensus was reached among all of the researchers 
for the coded transcriptions. Data saturation was achieved in that all of the qualitative 
responses could be categorized since all themes/codes from new data were already 
represented and that all of the categories were representative of the experiences of all the 
participants. Overarching themes from the codes were developed and similarities and 
differences in responses by respondents’ education sector (i.e., high school, 2-year, 4-year) 
were noted. Notes from the IBHE FAC meeting were incorporated into the qualitative 
results. While results from the focus groups and meetings cannot be generalized to all, 
they provide more in-depth information that is useful to inform the survey results. See 
Appendix C for a full description of the qualitative analyses procedures. 
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Results 


Demographics 

We surveyed 1,651 individuals and received usable responses from 373 state educators, 
resulting in a 23% response rate. The number of respondents for each question varied 
due to skipping patterns and missing data in the questions (see Table 1). 

Table 1: 


Number of Respondents Answering Survey Questions 


Number Surveyed 

1,651 

Usable Responses 

373 

Which Report Received 

373 

Received Both (123) 


Yr 1 only (47) 


Yr 2 only (42) 


Did not receive either report (161) 


Aware of report (160) 


Answered questions about Yrl report 

158-160 

Answered questions about Yr2 report 

148-150 

Have You Utilized the HS2CSR 

196 

Yes (125) 


Answered questions about utilization (121-124) 


No (71) 


Why not utilize (71) 


Utilized Report by Each Section 

112-116 

Which Sections are Most Useful (Ranking) 

106 

Extent Collaborated Using Report 

111 

Collaborating Inside/Outside Institution 

102 

Need for Executive Summary 

99 

Report Adds to Collaborations 

111 

Provided Example of Collaboration 


1st collaboration example (47) 


2nd collaboration example (33) 


3rd collaboration example (< 5) 


4th collaboration example (<5) 


Suggestions for Others to Receive 

25 

Any Additional Comments 

19 
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Table 2: 

Demographics of Survey Respondents 



Surveyed 

Responded 

% 

Average 
Years in 
Position 

Average 
Years in 
Education 

Total 

1,651 

373 

23% 

5.6 

23.6 

Principals/ Asst 
Principals 

600 

133 

22% 

6.2 

20.7 

Superintendents/ 
Asst Superint/ 
ROEs 

523 

119 

23% 

4.8 

27.2 

2-yr Administrators/ 
Faculty 

113 

74 

66% 

6.4 

23.0 

4-yr Administrators/ 
Faculty 

93 

23 

25% 

7.1 

27.2 

Workshop 

Participants* 

322 

24 


* 

- 


* Note: Workshop participants were recoded based on their response to “What is your title/role?’’ 


The majority of respondents were from the high school sector; however, there were fairly even proportions 
responding by each sector with the exception of a much higher response from 2-year senior administrators 
(66%) (see Table 2). This is likely due to the workshops being held on community college campuses and a high 
attendance rate of individuals from community colleges. The average number of years in the current position was 
5.6 and the average number of years employed in education was 23.6. There was roughly even representation of 
survey respondents from across the state, with a slightly higher response from individuals in the Northeast region 
(not including Chicago), and slightly lower response from the Chicago region (see Figure 1). The distribution 
of survey respondents closely mirrored that of the survey population, with a slightly higher survey response 
proportionally from the Southeast and Chicago regions (see Figure 1). 


Figure 1: 

Survey Respondents and Survey Population by Region 



Chicago 


Northeast Northwest East Central West Central Southeast 
(not including 
Chicago) 

Li Survey Population 


Southwest 
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Widespread 
dissemination of the 
HS2CSR was limited. 
While 33% received 
both reports , 43% of 
survey respondents 
reported not receiving 
either report. 


For the focus groups, 20 participants attended the eight focus groups, with 12 individuals 
from K-12 districts and eight from community colleges. Participants were located 
throughout the state (although mostly in the northeast region) and included principals, 
superintendents, workshop participants, and administrators from community colleges. 
The three face-to-face focus groups were held in the northeast, northwest, and east central 
regions of the state. The education meetings with the Chief Academic Officers included 
senior administrators from both the 2-year and 4-year public institutions. Individuals 
attending the IBHE FAC meeting included faculty from 2-year and 4-year institutions, 
representing both public and private institutions. 

Dissemination 

ACT disseminated the report with a cover letter from ISBE, IBHE, and ICCB through 
the US Postal Service to public secondary and postsecondary institutions. The reports 
were provided in a pdf format on a CD. High school reports were addressed to the 
principal(s) and superintendent of the K-12 district. Presidents/Chancellors of the 
community colleges and 4-year universities received their respective reports. ACT also 
sent the individual reports and an aggregate report to the three state education agencies, 
ISBE, ICCB, and IBHE, for posting on their agencies’ website. The high school reports 
were incorporated with the individual school’s report card information on the Interactive 
Illinois Report Card website (http://iirc.niu.edu). The first report (hereafter referred to 
as Year 1 report) was mailed in June 2011. The second report (i.e., Year 2 report) was 
mailed in February 2013. The reports were then made available to the public via the 
websites approximately one week after the ACT mailing. 

Survey respondents were asked to provide information about receipt of both Year 1 
and Year 2 reports, preference for receiving the report, and the effectiveness of the 
dissemination process. Respondents who indicated that they had not received nor read 
the reports were guided to an exit question which sought to gather information about 
other resources used. Those indicating having received or read the report were allowed 
to progress to the main section of the survey. 

As seen in Table 3, many of the respondents (43%) reported not receiving either report, 
suggesting a problem with distribution. One-third (33%) of the respondents indicated 
they received both reports; however, 13% received only Year 1 report and 11% received 
only Year 2 report. It should be noted that the Year 2 report was disseminated just one 
week prior to the online survey launch and that survey data collection ended one month 
later. Given that our survey invitations were sent to a wide array of individuals, some of 
whom were on the ACT mailing distribution list and others were not but were potential 
users of the report”, we examined if there were differences in dissemination between those 
on the ACT mailing distribution list and those not. As seen in Table 3, more individuals 
on the mailing list reported received both reports compared to those not on the list 


~ Those on the ACT mailing list included principals, superintendents, 2-year and 4-year presidents. 
Those not on the ACT mailing list but included in our survey included: regional superintendents, 
chief academic officers, enrollment directors, admissions directors, institutional researchers, and 
workshop participants. 
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Table 3: 

Receipt of Report: Overall, by ACT Distribution List and by Sector 



(39% versus 22%); however, a roughly equal percentage reported not receiving either 
report (42% versus 45%). The higher proportion of those not on the ACT mailing list 
and receiving Year 1 report only (22%) is due to the number of workshop participants, 
typically high school and community college faculty, who received the report at the 
workshops which were only offered after the Year 1 report release. We also see in Table 
3 that more K-12 sector individuals did not receive either report compared to those in 
the postsecondary sector (45% versus 33%). 

Respondents who did not receive either report (n=161) were asked if they were aware 
of the HS2CSR before receiving the online survey. Overall, 61% of these indicated 
they were not aware of the report prior to receiving the invitation to the online survey 
which included background information about the report, as well as links to the online 
reports. There were no differences in lack of awareness among the types of respondents: 
principals/assistant principals (61%), superintendents/regional superintendents (63%), 
2-year college administrators (54%), and 4-year college administrators (60%). Feedback 
from the focus groups indicated familiarity with the report was limited for many of the 
participants. One participant commented, 

“I’m pretty new to the report. . .1 actually had not heard of the report before. . .1 
[saw] an e-mail from IBHE saying there is an ACT High School to College 
Success Report. So then I happened to look at it and saw what information is 
provided in it.” (community college participant) 

In addition, most of those attending the meetings with the Chief Academic Officers 
and the IBHE FAC had not seen the report prior to the meetings. 

These results suggest a problem with the distribution strategy in that just over 40% of 
users, whether direct users or potential users, reported not receiving either report. And 
of these, 61% were not aware of the report. While we cannot say for certain whether 
the report reached its intended location, it appears audiences may not be aware of the 


Nearly two-thirds (61%) 
of those not receiving 
either report were aware 
of the report. 

“I’m pretty new to the 
report... I actually had 
not heard of the report. ” 
(community college 
participant) 
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report or may not be aware that the report was received in their office. Lastly, it is likely 
that the Year 1 report reached a slightly wider audience due to a longer time since its 
release date, attention garnered from the summit announcing the Year 1 report release, 
and training seminars held soon after the Year 1 report release. 


The HS2CSR was 
received from a variety 
of sources. Most 
respondents received 
the report in a timely 
manner and read it to 
some extent. The vast 
majority prefer email 
notification regarding the 
report availability. 


Next we examine differences in distribution between the Year 1 and Year 2 reports. As 
seen in Table 4, survey respondents received the report most often from one of three 
sources: state agency website, supervisor/administrator, and the ACT mailing. Slight 
differences were found with more respondents receiving the Year 2 report from the ACT 
mailing compared to the Year 1 report (38% versus 27%, respectively). Approximately 
seven out of ten respondents reported they received the reports in a timely manner (for 
both Year 1 and Year 2 reports). Despite the delay in disseminating the Year 2 report, 
only 8% indicated they received the report too late to use it. The vast majority (95%) 
of respondents preferred to be notified about the availability of the report by email, 
compared to weekly email newsletters by state education agencies, school newsletter, 
or by their professional organization. When we looked at notification preferences by 
education sector, we found that those in the K-12 sector also have a preference for the 
Superintendents online weekly messages (19%) and through their regional office of 
education (15%). 


Table 4: 


Dissemination of Year 1 and Year 2 Reports 



Year 1 
Report 

Year 2 
Report 

Received from (check all that apply) 

(n=160) 

(n=150) 

Agency Website 

42% 

37% 

Supervisor/Administrator 

28% 

29% 

ACT mailing 

27% 

38% 

Colleague 

11% 

7% 

Received in Timely Manner 

(n=158) 

(n=150) 

Yes 

68% 

72% 

Late, but still able to use 

21% 

20% 

Too late to use 

11% 

8% 

Extent Read Report 

(n=160) 

(n=148) 

Read full report 

18% 

15% 

To a great extent 

29% 

20% 

To some extent 

49% 

61% 

Not at all 

3% 

5% 

Notification Preference (check all that apply) 

(n=160) 

(n=150) 

Email notification 

95% 

95% 

ISBE Superintendent Weekly message 

14% 

17% 

ROE notification 

9% 

12% 

Professional Organization 

5% 

8% 

IBHE Friday Memo 

* 

* 

School newsletter 

* 

* 


Note: * indicates fewer than 5 respondents 
Percentages may not total 100 due to rounding. 
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Focus group participants reported receiving the report most often through email and 
on a state agency website, but also as a follow-up from participating in a regional 
workshop. When asked if they disseminated the report beyond themselves, many had 
not yet distributed the report but they had plans to do so. Some individuals planned 
to disseminate the results to their boards of education while a few others emailed notes 
about the availability and location of the reports to colleagues. A district superintendent 
described his/her dissemination strategy this way: 

“I usually share this with our principals, who disseminate this to our counseling 

ranks and through our department chairs to make sure that the College of 

Education’s features are embedded into our curriculum.” 

Respondents were asked to describe the extent to which they read each report. Nearly 
all of the respondents read the Year 1 or Year 2 reports to some degree (see Table 4) . The 
Year 1 report was read to a slightly greater extent than the Year 2 report (47% versus 
35%), with 18% indicating “read full report” and 29% indicating “to a great extent” 
compared to 15% and 20% for the same categories for the Year 2 report. This may be 
due increased publicity of the release of the Year 1 report, as well as the timing of the 
Year 2 report release and survey data collection. 
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Utilization and Usefulness: Overall 


Of those who received 
the HS2CSR, many 
have used it and have 
found it useful. 


Respondents provided information about their utilization of the HS2CSR. These 
questions sought to discover if individuals who received the report also utilized the 
report in their educational endeavors. Respondents were asked specifically whether they 
utilized the report, how they utilized the report, and which sections of the report they 
utilized. Additionally, respondents were asked to rate the report’s overall usefulness, as 
well as the usefulness of the sections they used. Respondents who indicated not utilizing 
the report were asked to provide reasons for not using the report. We first provide the 
results for overall utilization and usefulness and then take a closer look at the different 
sections of the report. 

Out of the 373 survey respondents, over half (57%) reported receiving either both reports 
or Year 1 only or Year 2 only (see Table 3). Of those who received the report, nearly 
two-thirds (64%; n= 125 3 ) utilized the HS2CSR. Figure 2 shows how often respondents 
used the report and to what extent they found it helpful. Of those who received the 
report, over half (64%) reported using the HS2CSR “occasionally;” although, only 
16% of these individuals used the report “frequently” or “very frequently.” Of those 
who used the report, 18% indicated the report overall was “very useful” and another 
49% indicated “moderately useful.” 


Figure 2: 

Of Those Who Received HS2CSR (n=125) 


Used Report 

n = 125 


Found Report Useful 

n=125 




1 N=196, since 17 respondents skipped “Have you utilized report?” question. 
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When asking about potential uses of the report, we provided a number of options of 
which the respondents could select all that applied to them (see Figure 3). Respondents 
indicated they primarily used the report for monitoring student success (63%) or 
collaborating within their institution (61%). Other reasons for using the report included: 
making curriculum/course changes (35%), collaborating outside their institution 
(31%), making program changes (29%), and making changes in advisement (19%) (see 
Figure 3). When we compared uses of the report by education sector, we found that 
individuals in the K-12 sector used the report more often for internal purposes (e.g., 
monitor students [69%], curriculum changes [50%], program changes [39%]) while 
the postsecondary sector was more likely to use the report when collaborating outside 
their institution (47%) (see Figure 3). 

Figure 3: 

Utilization of High School to College Success Report 



success my institution changes outside my advisement 

institution 

c5 □ Overall ■ K-12 □ Postsecondary 

(n=124) (n= 80) (n=38) 


Many of the focus group participants echoed the survey responses. Using the report 
to track student growth, to make changes and develop interventions, and to share the 
information was mentioned most often. Participants also indicated data on course taking 
patterns, GPA comparisons, and where students attend postsecondary institutions were 
most helpful. These comments are representative of their remarks: 

“It will help to use the changes in curriculum to see how our students are 
performing.” (high school participant) 

“The one piece that was very helpful to us was the part where it identified the 
actual courses, especially for math when we were trying to speak to our students 
[and] the parents about college readiness.” (community college participant) 


The HS2CSR is 
primarily used to monitor 
student success and 
to collaborate within 
institutions. 


Members of the IBHE FAC also mentioned curriculum alignment and developing 
interventions (e.g., workshops for students in developmental education, offer ACT 
as early indicator for college readiness) as potential uses of the HS2CSR. In addition, 
individuals suggested using the report to facilitate dialog between sectors, to address 
advising issues, and to assess high school interventions. A couple of specific research- 
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oriented suggestions were provided: 1) investigate reasons related to students taking 
more math in high school (e.g., family income, family support); and 2) examine 
why high school students are not completing courses identified as needed to succeed 
in postsecondary education. Attendees at the CAO meetings also agreed the report 
facilitates conversations between secondary and postsecondary about various topics such 
as curriculum alignment or remediation, “ [The report] is a starting point. People could 
take a look at it and see if the students have met the standards or not.” 

Reasons for not using. Respondents who do not utilize the report (36%; n=71) 4 were 
asked to provide insight into why they are not using the HS2CSR (see Figure 4) . The 
most frequent (49%) response included not having enough time to review the report 
but intended to do so in the future. Additional reasons for not using the report involved 
issues with the data and readability of the report, namely: report is confusing (21%), 
data not reliable (17%), better resources exist (15%), data is not relevant to work (14%), 
too much information (13%), and data is outdated data (10%). There were too few 
responses to make comparisons by sector. 


Lack of time is the 
primary reason for not 
using the HS2CSR. 

“It would be much more helpful if it also included some private schools and 
if it was a little more comprehensive.” [high school participant) 

“I realized the report was not something we could use because it’s such a small 
sub-segment of our students.” [community college participant) 


Similar to the survey respondents, several focus group participants expressed concern 
about the data such as comprehensiveness of the data, reliance on self-report data, and 
using ACT instead of GPA as a predictor of college success. Others were interested in 
gaining a better understanding of the data before utilizing the report. Here are several 
comments that represent the concerns: 


Figure 4: 

Reasons for not using the High School to College Success Report 



No time but Report is Data are not Better Other Data is not Too much Not enough The data is 

plan to utilize confusing reliable resources relevant to information information outdated 

exist my work 

n=71 


4 


N=196, since 17 respondents skipped “Have you utilized report?” question. 
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“The report now, as it stands, tells me first-time, full-time only, and, so it does 
not give a complete picture of all. It certainly doesn’t even touch the adult 
students, but even traditional students who might be coming in part-time or 
for summer, or swirling students.” ( attendee at CAO meeting ) 

“I think first, what would be helpful is more understanding of how to use the 
report... I think for me to be able to use it, I really need to know what I can use it 
for and that is not really clear for me right now.” ( community college participant) 

Utilization and Usefulness: By Report Section 

To drill down into more specific feedback about the report, we asked respondents to 
indicate their utilization and perceived usefulness of the different sections of the report. 
Respondents also ranked the sections of the report in terms of usefulness to prioritize 
their preferences. Summaries of respondents’ suggestions to improve specific report 
sections are included later in this report. 

As seen in Table 5, the College Readiness and Success section was used by the most 
respondents (90%), followed closely by the High School Preparation and Success section 
(88%). The Illinois Custom Addendum was used by the fewest respondents (65%). 
We asked respondents to indicate to what extent on a 5-point Likert scale (strongly 
disagree to strongly agree) they agree that the specific section was useful. As seen in 
Table 5, the percentages ranged from 75% to 82% of respondents indicating they 
“agree” or “strongly agree” that the specific section was useful; thus, all of the sections 
were viewed as nearly equally useful by the respondents. We then asked the respondents 
to rank order the sections to help prioritize the value they placed on the data. In terms 
of ranking the sections by usefulness, the High School Preparation and Success section 
was ranked the highest, followed in order by College Readiness and Success, College 
Success and Persistence, Detailed Summary Information by Campus, and lastly Illinois 
Custom Addendum. These rankings closely align with the respondents’ reported use of 
the section as well as their opinion about usefulness. 


The High School 
Preparation and 
Success section was 
ranked the highest for 
usefulness. 


Table 5: 


Utilization and Useful of Report Sections: All Respondents 



Overall 


Report Section 

Yes, Used 
this section 
(n) 

“Agree” 
is useful 

“Strongly 
Agree” is 
useful 

Rank 

High School Preparation and Success 

88% 

(n=102) 

76% 

5% 

1 

College Readiness and Success 

90% 
(/7=1 03) 

67% 

13% 

2 

College Success and Persistence 

82% 
(n= 93) 

69% 

9% 

3 

Detailed Summary by Campus 

77% 
(n= 87) 

74% 

8% 

4 

Illinois Custom Addendum: Charts 

65% 

ip-72) 

64% 

11% 

5 

Illinois Custom Addendum: Tables 

65% 
(n= 73) 

67% 

8% 

5 
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Table 6: 

Utilization and Useful of Report Sections : By Sector 



By Sector 



Yes, Used this 
section 

“Agree” 

Agree 

or “Strongly 
’ is useful 

Rank 

Report Section 

K-12 

Postsec 

K-12 

Postsec 

K-12 

(17=7?) 

Postsec 

(17=34) 

High School Preparation and Success 

90% 

(n=70) 

84% 

(n=31) 

81% 
(n= 56) 

80% 

(17=24) 

1 

3 

College Readiness and Success 

91% 

(n=70) 

89% 

U=32) 

80% 
(n= 56) 

78% 

(77=24) 

2 

1 

College Success and Persistence 

81% 

(n=62) 

83% 
(i7= 30) 

76% 

(77=47) 

80% 

(17=24) 

3 

2 

Detailed Summary by Campus 

74% 
(n= 56) 

83% 
(n= 30) 

80% 

(n=44) 

87% 

(17=26) 

4 

4 

Illinois Custom Addendum: Charts 

61% 

(n=46) 

74% 
(77= 26) 

74% 

(n=34) 

77% 

(17=20) 

5 

5 

Illinois Custom Addendum: Tables 

61% 

(n=46) 

74% 

(77=26) 

74% 

(17=34) 

77% 

(17=20) 

5 

5 


Postsecondary 
respondents rank the 
College Readiness and 
Success section the 
highest for usefulness. 


Table 6 breaks down the utilization and useful results by K-12 and postsecondary 
sectors. When comparing use by sector, we see fairly even distribution for most of 
the sections, with the exception of more postsecondary respondents using the Illinois 
Custom Addendum Charts and Tables (74% versus 61% for K-12 respondents). We see 
only small variations in the percentage of K-12 respondents compared to postsecondary 
respondents who “agree” and “strongly agree” that the report sections are useful. As 
might be expected, we see slight differences in the rankings when broken down by 
sector. The postsecondary sector ranked the sections pertaining to College Readiness and 
Success and College Success and Persistence higher than the High School Preparation 
and Success sections. 


Collaboration 


Understanding how educators are using the HS2CSR in collaborations with others 
was one of the primary focal points of the evaluation. The survey sought to gather 
information on not only the extent of collaborations using the HS2CSR but also whether 
participants are collaborating inside and outside of their institutions and how useful 
the HS2CSR has been in contributing to the collaborations. The survey also explored 
possible barriers, as well as aspects enhancing collaborative efforts. Survey respondents 
also provided information on how the HS2CSR has changed or, in the future, might 
change collaborative activities to improve student success. Furthermore, information 
was collected on other data/information educators are using in their collaborative efforts. 
Finally, details about specific examples of collaborative relationships were requested. 
Survey respondents who reported not collaborating through use of the HS2CSR were 
invited to provide reasons. Respondents’ recommendations for others who would benefit 
from receiving the reports are included later in this report on suggestions for improving 
the HS2CSR. 
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Collaboration among participants. Survey respondents were asked to indicate the 
extent of collaborative efforts using the HS2CSR on a 4-point Likert-type scale ranging 
from “never” to “often.”Overall, we found survey respondents are using the HS2CSR 
in their collaborative activities, they find it helpful, and they feel that the report adds to 
their collaboration discussions (see Table 7). The majority of respondents indicated they 
“sometimes” (58%) collaborate using the HS2CSR. On the other hand, others reported 
they “seldom” (32%) or “never” (8%) use the HS2CSR in their collaborative activities 
(n= 111). Collaborations using the HS2CSR to improve student success occurred more 
often with individuals within their institutions (51% “sometimes” and 41% “often”), 
compared to those outside their institutions (38% “sometimes” and 12% “often”). Nearly 
two-thirds (62%) of survey respondents agreed that the HS2CSR has been helpful with 
inside collaborations, compared to 40% agreeing the report was helpful for outside 
collaborations. Anticipating that other reports and data are used by educators, we asked 
to what extent the HS2CSR adds to their current collaborative efforts to improve student 
success. We found that the HS2CSR helps “some” (74%) and “quite a bit” (15%) ( n= 
111) above and beyond other reports and data used by the survey respondents. 


Survey respondents 
are using the HS2CSR 
in their collaborations , 
more often within their 
institutions. 


Table 7: 

Collaboration Activities using High School to College Success Report 

Collaborate Using HS2CSR (n=111) Extent HS2CSR adds to collaborations (n=111) 


Often * An extreme amount 






Collaborated within your Institution 

(n=102) 

Collaborated outside your Institution 

(n=102) 


Strongly Agree 

* Fewer than 5 respondents 


Never 1 3% 


Rarely 

37% 

Sometimes 

38% 

Often 

12% 

HS2CSR helpful for external 
collaborations 

(n=102) 

Strongly Disagree 

* 

Disagree 

10% 

Neither Agree nor Disagree 

46% 

Agree 

40% 


Never 0% 


Rarely 

8% 

Sometimes 

51% 

Often 

41% 

HS2CSR helpful for internal 
collaborations 

(n=102) 

Strongly Disagree 

* 

Disagree 

•k 

Neither Agree nor Disagree 

25% 

Agree 

62% 


Strongly Agree 9% 


None 

9% 

Some 

74% 

Quite a bit 

15% 


Never 

8% 

Seldom 

32% 

Sometimes 

58% 


When examining collaboration by sector, we collapsed response categories into “never/ 
rarely” and “sometimes/often” due to small cell counts. As seen in Table 8 {next page), 
we find that the K-12 sector is collaborating using the HS2CSR more often than the 
postsecondary sector (67% versus 49% for “sometimes/often”). This may likely be due 
to the postsecondary institutions’ access to data or the K-12 sector’s recent focus on 
implementing the Common Core State Standards. No differences were seen between 
sectors for collaborating inside their respective institutions (over 90% for each sector 
for “sometimes/often”). Similar to the results discussed above regarding uses of the 
HS2CSR, we find that more individuals in the postsecondary sector are collaborating 
outside their institution compared to the K-12 sector (70% versus 42%). 
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Table 8: 

Collaborations by Sector 


Collaborating 
Using HS2CSR 

“Somtimes/Often” 

K-12 

67% 
(n= 50) 

Postsecondary 

49% 
(n= 17) 

Collaborating 

Inside 


Institution 

“Somtimes/Often” 

K-12 

90% 

(r?=64) 

Postsecondary 

over 90% 
(small cell count) 

Collaborating 

Outside 


Institution 

“Somtimes/Often” 

K-12 

42% 
(n= 30) 

Postsecondary 

70% 

D=21) 


Barriers to Collaboration. We asked survey respondents broad 
open-ended questions (i.e., not focusing specifically on the HS2CSR) 
to tell us about barriers they have experienced in collaborating with 
others to improve student success. We also asked what facilitated their 
collaborations. These questions were also addressed in the focus groups. 
Regarding barriers to collaborations, several themes emerged, namely: 
1) constraints of time and resources, 2) differences in institutions’ 
mission, 3) concerns with the report, and, 4) fear of offending someone. 
Lack of time and resources was cited most often by survey respondents, 

“I’m going to be honest with you; most principals if they are like 
me don’t have a lot of time.” ( high school participant) 

Related to time is the challenge of finding a time that fits the 
institutions’ differing schedules, 

“...we run on different schedules. Especially when you’re trying 
to do this data sharing and helping on a faculty-to-faculty level 
collaboration, it can be hard to find times that the community 
college faculty can be away that will align with the best times for 
the high school faculty or middle school faculty to be able to be 
away.” ( community college participant) 


The postsecondary 
sector collaborates 
more often outside their 
institution. 


Many respondents indicated different missions and priorities can create tension between 
collaborators: 

“Different cultures make direct sharing of concepts and ideas difficult.” ( high 
school respondent) 


Several individuals remarked being concerned about using the HS2CSR in collaborating 
when they felt the data were not representative of their schools’ student population or that 
the data were self-reported. One comment from the focus groups reflected this concern: 


“The thing is if I sent 40 kids off to college, I’m only getting data back on 8 
of them. I don’t consider it very complete data and I don’t know if. . .1 want to 
make a decision based on [that data].” ( high school participant) 


Barriers to 
Collaboration 

• Time constraints 

• Differences in mission 

• Concerns about report 
data 

• Fear of offending 
someone 


Several focus group participants also indicated it was hard to compare GPA across 
different institutions: 

“Different schools for different reasons do different things and it’s hard then 
to get a good normed comparison group going with neighboring districts or 
even within schools in the same district.” ( high school participant) 

Lastly, several survey respondents indicated concerns about offending someone when 
discussing the data: 


“It is easy to become defensive for some individuals/universities when they 
see this data.” ( high school respondent) 


“There is sensitivity to discussing these results with individual school districts 
for fear of seeming accusatory.” ( community college respondent) 
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Facilitators to Collaboration. Regarding what facilitated collaborations, survey 
respondents’ feedback centered around three themes which were evenly represented: 
availability of data, existing relationships, and a willingness to work together. Many 
respondents indicated having data available (particularly in an electronic format) about 
their students is needed and adds to their collaborative discussions: “A variety of data 
resources and this report is just one of many” and “The recent Common Core Standards 
mapping has increased the collaboration between the community college and high 
schools. The conversations about Programs of Study have also allowed more candid 
conversations about student preparation and success to take place.” Many of the survey 
respondents indicated having existing relationships within and outside their institutions 
to address college readiness, as well as for other initiatives (e.g., Perkins Program). Much 
of the focus has been on curriculum alignment among faculty in the high schools and 
community colleges. One survey respondent remarked, “Collaboration with others will 
help students be more prepared academically, financially, and socially.” A willingness to 
work in partnerships was also a sentiment expressed by many survey respondents: “A 
commitment towards providing students with quality post-high school opportunities and 
preparing them for future aspirations,” “A common interest in improvement,” and, “A 
willingness to work together to improve student success.” Lastly, one survey respondent 
commented on the benefit of outside groups facilitating collaborations: “The assistance 
of ISBE, ICCB, IBHE, and the ROE’s (Regional Office of Education) assistance and 
collaboration has been helpful. ’’Members of the IBHE Faculty Advisory Council also 
indicated sharing information and having discussions between K-12 and postsecondary 
would facilitate collaborations, particularly around curriculum, assessment, and grading 
alignment. The discussions should include high school and college-level counselors, 
faculty in 1 st Year Experience committees, and faculty from secondary and postsecondary 
institutions. Lastly, it was suggested to re-energize the Alliance for College Readiness 
models statewide. 


Facilitates 

Collaboration 

• Availability of data 

• Existing relationships 

• Willingness to work 
together 


“A willingness to work 
together to improve 
student success. ” (high 
school respondent) 


How HS2CSR changed collaboration activities. Many respondents noted that the 
HS2CSR has changed efforts to improve student success, while a few respondents 
mentioned they had no changes planned or that the data would be ignored. Among 
these efforts were changes in curriculum (alignment, creating a core, etc.) and pedagogy, 
student programming, setting smarter goals, increasing recruitment and retention 
efforts, closing the achievement gap, and using the report as an additional data tool. 
Comments included, 

“It has modified the way that we instruct, recruit and align curriculum. We 
are currently working on a First Year Experience class to address some of 
the district’s student issues. Retention and completion is our ultimate goal.” 

( community college respondent) 

“It is an additional data point referenced to change instruction to include more 
rigor and engagement.” ( high school respondent) 
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HS2CSR is viewed as 
one more data tool in 
a long list of available 
resources. 


A few respondents stated that the HS2CSR itself will change their collaboration activities 
because they already utilize similar data: 

“We are already incorporating these data in our curriculum alignment, 
Common Core State Standards, and student success discussions with all of 
the high schools in our district.” (high school participant) 

Several respondents commented on the reports potential: 

“This will not only be helpful with Common Core State Standards 
implementation, but also as we prepare for a new evaluation system for teachers 
and administrators.” ( high school participant) 

“This report has the potential to be an important component of our 
improvement efforts — as we shift to Common Core Standards and embrace 
the college/career readiness benchmarks — the report will be very important 
to us.” ( high school participant) 

Other data/information resources used in collaborations. With the increased focus 
on data-informed decision making, we asked respondents to tell us about other data 
and information resources used in their collaborative efforts to improve student success. 
As seen in Table 9, educators are utilizing many different types of data from many 
different data sources. Far and above all other items, utilizing the Common Core State 
Standards to align curriculum was cited most often by survey respondents. Some of the 
other frequently mentioned data or data sources included: locally developed assessments, 
NWEA MAP, ACT EPAS, ACT, and the Interactive Illinois Report Card. Focus group 
participants reiterated many of these items, particularly ISAT, PSAE, ACT, and ACT 
COMPASS. Several focus group individuals from both high schools and community 
colleges also mentioned using internally generated data. A K-12 survey respondent 
stated that a new resource within the district was a newly created position that focuses 
on student retention and success. As noted above in respondents’ comments, the 
HS2CSR is one more data tool of a long list of data resources to inform discussions 
about improving student success. 

Examples of Collaborative Activities 

In order to further examine the collaborative efforts using the HS2CSR, we asked 
survey respondents to describe detailed examples of their collaborative activities and 
to identify their collaborator(s) in terms of their education sector (i.e., high school or 
district, Regional Office of Education, community college, and 4-year universities). 
Respondents could select “all that apply” when identifying their collaborators for a 
specific example. Many individuals provided rich examples of their collaborations: 47 
survey respondents described one collaborative example, 33 described two examples, 
2 described three examples, and lastly, 2 described four examples (which was the highest 
number described). We identified and summarized themes from their descriptions below. 
See Appendix D for verbatim responses. 
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Table 9: 

Resources Used in Collaboration 

Local Data Sources 

Advanced Placement (AP) participation/performance 
records, 

Annual databook for college administrators 
Attendance 

Community college reports similar to the HS2CSR 
Community college admissions testing 
Curriculum maps (locally developed) 

Discipline behavioral data 
Failure rates 

Feedback reports to high school counselors 

GPA 

Grades 

Fionors courses taken 
Incoming student demographics 
Individual student tracking goals 
Interim formative assessment scores 
Internal feedback report for high schools 
Interviews with graduates after their first semester of 
college 

Locally written assessments 
Parental involvement information 
Persistence and completion records 
Placement tests 

Programs/services offerings on campus surveys 
School report card database 
Student placement 
Student work analysis 

Trend data and comparisons with similar schools 
Yearly growth data for students 

S tandardized Assessments 

ACT 

ACT COMPASS 

ACT Educational Planning and Assessment System 
(EPAS) 

ACT EXPLORE 
ACT PLAN 
ACT WorkKeys 
Achieve 3000 Lexile Scores 
Achieving the Dream 
Aimsweb data 

Community College Survey of Student Engagement 
Charlotte Danielson framework 
College Student Inventory (CSI) 

Common Summative Assessments 
Common Formative Assessments 
Discovery Education Assessment 
Developmental Reading Assessment (DRA) 

Illinois Standard Achievement Test (ISAT) 

KeyTrain 


National Student Clearinghouse 
Northwest Evaluation Association - Measurement of 
Academic Progress (NWEA MAP) 

Positive Behavior Interventions and Supports(PBiS) 
Prairie State Achievement Examination (PSAE) 

Read 180 

STAR Math/Reading Testing Program (Renaissance 
Learning) 

Study Island 

School Wide Information System (SWIS) 

ThinkLink 
Voyager resources 

Additional Resources for collaborating 

Alignment with Common Core State Standards 
Benchmark testing 
Dual credit collaboration 
Early Child performance data 
Chooseyourfuture.org 
College Greenlight website 
Complete College America meetings 
Docufide by Parchment 
Great River Economic Development Group 
Illinois Articulation Initiative - General Education 
Course Curriculum (IAI GECC) 

Illinois Association for College Admissions Counseling 
(IACAC) 

Illinois College Access Network 
Illinois Community College Board data 
Illinois Student Assistance Commission 
Interactive Illinois Report Card (IIRC) 

Integrated Postsecondary Education Data System 
(IPEDS) 
i-Transfer.org 
Kaplan materials 
Ladder Up Organization 

National Center for Education Statistics (NCES) 
National Community College Benchmark Project 
New Teacher Evaluation Tool 
Professional development through local districts, 
Regional Office of Education, and ISBE 
Regional Offices of Education materials 
Research data from education research field 
Retention committee at postsecondary level 
Rising Star School Improvement Plan 
School improvement plan 
Statewide System of Support (SSOS) coach and 
resources 

State and federal initiatives and directives 
Voluntary Framework for Accountability 
What’s Next Illinois website 
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Sharing the connection 
between ACT 
performance and college 
performance with their 
students was “powerful. ” 
(high school respondent) 


Considering the education sectors of all the collaborators identified (n= 208), we found 
that the majority of collaborators (56%) were from the high school or district level. 
Twenty-eight percent (28%) of the collaborators were from community colleges. Fewer 
collaborators were at the ROE level (9%) and 4-year university level (7%). These results 
could be due to the fact that our survey respondents were overwhelming from the 
K-12 sector. In addition, the K-12 sector has been impacted more from the recent state 
initiative of implementing the Common Core State Standards. 

Several themes emerged from the survey respondents’ detailed descriptions of the 
collaborative activities, namely sharing data with others, internal collaborative activities, 
and external collaborative activities. 

Sharing Data. First, several respondents described disseminating the HS2CSR 
data to other interested groups, such as school boards, counselors, high school 
teachers, parents, and students. One respondent remarked that sharing the 
connection between ACT performance and college performance with their 
students was “powerful.” Another reported sharing the data with high school 
departments “so that they could see linkages between their instruction, ACT 
scores, and college success rates.” 

Internal Collaborations. Several of the examples described collaborations 
within the school/college to address specific issues such a college persistence or 
college readiness, or to utilize the information for the school’s improvement plan. 
One illustrative example included: “The district and high school administrative 
teams began to collaborate. The Superintendent led pipeline discussions with the 
entire K-12 administrative team to build the pathway for accelerated progress 
and increasing expectations from K-12. Out of these discussions came staffing 
changes, co-curricular programs after school and in the summer that went 
beyond at-risk student populations, and increased AP training, course offerings, 
and enrollments.” 

External Collaborations. The last theme centered on collaborations across 
institutions. These collaborations dealt with topics such as increasing 
communication, increasing postsecondary placement scores, dual credit, and 
curriculum alignment (e.g., math, English, computer science and engineering). 
Examples of the cross-institution collaboration included: “An Associate Dean 
of mathematics and several math faculty members have been working with 
high school faculty members to align the curriculum. Their goal is to close the 
expectation gap between high school math and the college placement exam,” 
and “Community college and high school counselors’ meetings to discuss 
curriculum, gaps and misunderstandings.” Lastly, one respondent reported a 
negative experience with collaborating with a college that had not been successful 
due to unions and administrative turnover. 

Feedback from the focus groups was very similar in terms of the types of collaborative 
activities. Several mentioned using the report for curriculum decisions and conversations 
about college readiness. Utilizing the HS2CSR in collaborations across education sectors 
among principals, superintendents, and faculty at both the secondary and postsecondary 
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levels were also described. We asked a follow-up question whether the collaborations 
had been occurring prior to receiving the report. Of the respondents that were asked 
this question, nearly all reported collaboration relationships were present prior to the 
report. Below are examples of collaborative activities obtained from the focus groups: 

“I think the whole college readiness, the Common Core Standards, was another 
good content that helped us provide an event and a connection to an invitation 
again to get together.” ( community college participant) 

“We have been reaching out to our high schools. In fact, members of our English 
department are currently adopting a local high school, so that we can work 
with them. What we are doing is looking at college writing samples that are 
adequate to place into college -level English. We are very pleased with the way 
this is going and with the spirit of collaboration that’s emerged.” ( community 
college participant) 

“We had a relationship with our community college prior to me ever seeing 
this report but we’ve always used data collected by them and data collected 
by us. And then we have always had questions about whose data was actually 
correct. So, I actually like this report because we can both look at the same 
thing and also be able to draw conclusions to how to help our students work 
better.” ( high school participant) 

Respondents’ Suggestions for Improving HS2CSR 

Throughout the survey we asked respondents to provide feedback to improve the 
HS2CSR. We asked for suggestions for additional data elements to include and specific 
data elements to eliminate that will increase the usefulness of the report. We asked for 
suggestions for the individual sections of the report. We also asked for suggestions to 
improve the dissemination of the report and if a summarizing brief of the report would 
be helpful. The sections below provide a summary of the responses. Appendix E provides 
the verbatim responses for each of the sections separately. We also asked focus group 
participants for suggestions. Their comments are incorporated below. 

Overall. Several individuals provided global positive feedback about the usefulness of 
the report: “I feel the report is all very useful” and ‘The report is very comprehensive.” 
A few commented that the report, although helpful, revalidates what educators “have 
known for years” that students taking advanced classes score higher on ACT and are 
more prepared for college. Another commented, “The report helps affirm pre-existing 
notions and ideas and helps verify what is already known.” Lastly, a couple of survey 
respondents remarked that they have not used the HS2CSR “as fully as I should.” 

Increase accuracy. Although many respondents provided positive feedback about the 
HS2CSR, many have significant concerns about the comprehensiveness of the data that 
inhibited their use of the report. Others had concerns about the data; however, they 
still found the report useful as long as they kept the limitations in mind. Respondents 
indicated the lack of data from private institutions and out-of-state institutions reduced 
the usefulness of the report because the data were not an accurate representation of 
the students in their schools, especially when their high-performing students typically 
attended out-of-state institutions. This comment is representative of many respondents’ 


“I actually like this report 
because we can both 
look at the same thing 
and also be able to draw 
conclusions to how to 
help our students work 
better. ” (high school 
participant) 


“The report only contains 
information on students 
that attend Illinois public 
institutions. Inclusion 
of all postsecondary 
schools would be a 
more complete picture. ” 
(high school respondent) 
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Increase Ease of Use 

• Provide definitions 
and data sources 

• Provide training 

• Provide data in an 
interactive, electronic 
system 


“The ability to 
manipulate the data 
electronically, so that it’s 
more meaningful to us, 
would be very helpful. " 
(community college 
participant) 


opinion: “The report only contains information on students that attend Illinois public 
institutions. Inclusion of all postsecondary schools would be a more complete picture.” 
Several respondents expressed concern about using self-reported data and questioned 
the accuracy of these data. Another concern was the use of GPA rather than grades (i.e., 
A, B, C, etc. were used in the former High School Feedback report). While GPA allows 
for numerical comparisons (e.g., 2.9 vs 2.7), grades provide a range for measurement; 
and in doing so, “we have gained simplicity but not sure we gained meaning.” Lastly, 
a couple of respondents had strong negative opinions about the accuracy of the ACT 
benchmarks. 

Increase ease of use. Respondents from focus groups and meetings suggested that more 
clarity within the report would improve its usefulness, such as providing definitions 
and calculations for data measures and identifying data sources. Comments such as 
“What is general science?” and “How is development education defined?” indicate 
more detailed descriptions are needed. Providing training and examples of how other 
schools utilize the data would also increase understanding and the likelihood of utilizing 
the report. Local regional offices of education could be utilized to provide the training 
since they already have established relationships with the local school districts. Both 
survey respondents and focus group participants expressed interest in face-to-face 
training or webinars. Many preferred face-to-face training, “Well personally, I need 
to sit down... being able to hear what other people are doing, see what other people 
are doing is most beneficial” (high school participant)-, however, others commented that 
webinars are more cost efficient and flexible, “It doesn’t matter to me if it’s face-to-face 
or webinar as long as there are multiple opportunities. . .It’s hard to find funding to 
transport people” ( community college participant) . Several respondents from the survey, 
focus groups, and meetings, suggested providing the data in an interactive, electronic 
system so that stakeholders would have more flexibility in using data and incorporating 
it with their local data systems. A number of focus group participants remarked, “The 
ability to manipulate the data electronically, so that it’s more meaningful to us, would 
be very helpful” ( community college participant) and “If there was some kind of electronic 
way to look at the data that would be more helpful than waiting for the report. I think 
it would be probably used more widely” ( community college participant) . A high school 
participant also commented about the usefulness of an interactive report, “I don’t want 
data for data’s sake but if I can utilize it and manipulate it and draw some questions 
out from our own organization’s perspective, it’s a good thing.” Another suggestion 
from the CAO meetings was to provide in the “digital format of the report the ability 
that the table of contents link to the respective areas of the report to reduce time spent 
searching for information.” 

Provide additional data elements. Numerous and varied suggestions pertaining to 
additional data were provided. Several respondents indicated all data records should be 
reported “regardless of a small count.” It was remarked that excluding this information 
“ . . .limits the use of information for small high schools.” One suggested compromise of 
lowering the limit to five (rather than 10) was offered. Several respondents suggested that 
the report should include more comparative data. For instance, some respondents felt that 
comparing similar schools (e.g., geographical locations, region, funding levels, specific 
institutions) would strengthen the overall report. A few survey respondents requested 
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that individual students’ names be included in order for them to match with their 
local data for further investigation; although, feedback from the IBHE FAC members 
discussed balancing the need for confidentiality with the need for comprehensive data. 
Several respondents suggested the data be disaggregated further by geography, minority 
or underrepresented populations, separate graduation years, demographic variables, and 
different GPA and ACT categories. Other specific suggestions included: 

• more recent data 

• track data over time 

• keep archived reports online 

• college course enrollment 

• track students from community college to university level 

• more information on developmental math 

• other variables that impact student success (e.g., financial information) 

• more information about postsecondary schools where students attend (e.g., 
entrance exams). 

Focus group participants and meeting attendees repeated several suggestions for 
additional data, namely, trend data, comparisons with other groups or schools/ 
institutions, demographic breakdowns, and data for all of their students (e.g., part- 
time, adult students, delayed-starters) . For example, the IBHE FAC members felt 
that better comparison groups (other than just the entire state) would be helpful and 
suggested using national data for comparison. Additional data elements to consider 
included: dual-credit enrollment, major, high school rank, socio-economic status, time 
of registration for college, breakout data for students with documented disabilities, 
intentions for postsecondary enrollment, and data for where non-persisters go after they 
drop out (e.g., out-of-state, 2-year, 4-year, employment). Several individuals expressed 
interest in tracking students from the community colleges to the 4-year institutions. 
Other subgroups to consider included veterans and later-adult re-entry cohorts. An 
IBHE FAC member suggested examining the impact of science course taking patterns 
on GPA in different postsecondary majors, programs, or institutions. One focus group 
attendee commented, 

“I think it would be also nice to have some types of trends over the years that 
are put on each one so that I don’t have to say well how does this compare 
to last year and develop my entire own data warehouse spreadsheet of it so I 
compare 2011 to 2012 to 2013.” ( high school participant) 

An attendee at a CAO meeting commented, 

“There was also indication that a more longitudinal analysis would be more 
helpful because some of the data is very limited in the amount of students 
included. The HS2CSR does not help with examining existing programs within 
the universities but it was the hope that the longitudinal data system might 
help link data to specific programs and not just to high schools.” 


HS2CSR should 
include all data records 
“regardless of a small 
count. " Otherwise it 
“limits the use for small 
high schools. ” (high 
school respondent) 
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An executive summary 
would be helpful in 
disseminating the report 
and in understanding 
the data. 


A high school 
participant commented 
on the potential of 
the HS2CSR: “Once 
courses are aligned to 
the core competencies 
this data will be more 
valuable. At present it is 
difficult to understand at 
school level because the 
courses they take have 
differences (i.e., English 
1A, IB , and 1C).’’ 


Executive Summary. In terms of improvements, many suggested providing an executive 
summary to help understand and disseminate the information to other stakeholders who 
are less familiar with the report (e.g., school boards, parents, faculty). Responses to our 
survey question about a summarizing brief corroborated this suggestion. Three-fourths 
of the respondents “agreed” (59%) and “strongly agreed” (17%) that a summarizing brief 
(2-3 pages) would aid in understanding and using the report data, as well as facilitate 
collaborative endeavors. Nearly all of the focus group participants (both high school and 
community college) indicated an executive summary would be helpful, particularly when 
sharing with school boards of education and others who may not be as familiar with the 
data. One participant felt it would be helpful given the large volume of information in 
the report, “It gets around some of the problems that I find with just the length of the 
report. It needs something like that badly” ( community college participant) . A comment 
received from the IBHE FAC concurred, “An executive summary with just two or three 
key data charts would be useful as an orienting piece so readers don’t have to dig through 
pages and pages of charts.” 

Dissemination Suggestions. When asked who else might benefit from receiving the 
report, several survey respondents provided suggestions: middle school administrators 
and faculty, high school guidance counselors, high school curriculum directors, 
Education for Employment (EFE) Directors, directors for student services/guidance, 
high school test coordinators/data analysts, deans/directors in the community college, 
and vice-president/ chancellors at 4-year institutions. Focus group participants also 
suggested school board members, parent and community groups, school improvement 
teams, guidance counselors, curriculum coordinators, and the Chicago Area Directors 
of Curriculum (CADCI). Suggestions from the CAO meetings included sending to 
enrollment and institutional research offices and distributing through the CAO’s listserv. 
Interestingly, participants from both the survey and focus groups suggested individuals 
who are currently on the distribution list. Providing a list of those on the distribution 
list (e.g., high school principals and superintendents, 2-year and 4-year presidents) along 
with the report, as well as specifically stating the report should be disseminated to others 
in their institutions would help inform every one of the dissemination strategy, and 
possibly encourage collaboration. A couple of suggestions from the open-ended survey 
questions included utilizing the ICCB to disseminate the report to community colleges 
and posting the current report on the agency websites as soon as they are available. 

High School Preparation and Success section. Several of the responses were similar 
to those stated above, such as provide definitions, put online in a relational database, 
track data trends, and provide more comprehensive data from private and out-of-state 
institutions. A couple of respondents remarked the data were “extremely useful” and 
indicated the report was good “as is.” Some specific suggestions included: 1) providing 
more detailed information (break out each chart by high school for the college report, 
separate graduation years); 2) comparing high schools with similar demographics; and, 
3) comparing developmental versus non-developmental students. Lastly one respondent 
commented on the potential value of the HS2CSR, “Once courses are aligned to the core 
competencies this data will be more valuable. At present it is difficult to understand at 
school level because the courses they take have differences (i.e., English 1A, IB, and 1C).” 
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College Readiness and Success section. Feedback for this section was somewhat limited. 
The need for trend data was reiterated. Some expressed concerns about using GPA data 
and self-report data, as well as the emphasis placed on standardized tests. A couple of 
respondents indicated they were using the data to reconsider their use of placement into 
developmental and college level courses and to inform their Enrollment Management 
Plan. Lastly, one respondent expressed interest in more nuanced information, “What 
types of developmental coursework is needed by students specifically? Are there certain 
skills that are lacking?” 

College Success and Persistence section. Similar themes regarding data were echoed 
for this section, namely: providing definitions (persister vs non-persister), providing 
more comprehensive data with out-of-state and non-public schools, providing trend 
data, and providing comparison data with similar schools. Several respondents suggested 
additional breakdowns of the data: non-persisters by GPA ranges, persistence rates by 
different ACT ranges, rates of non-persisters by school, persistence rates by community 
colleges versus 4-year institutions, and unique student IDs to conduct follow-up. The 
new Illinois Longitudinal Data System will provide information for one respondent’s 
request regarding non-persisters and their subsequent enrollment patterns. Lastly, several 
respondents provided suggestions that would require additional data or a research study, 
for example, “Determine the cause for the students not remaining in college after the 
1st year. For example, cost of the school, lack of academic progress, social issues with 
the school, etc.” Members from the IBHE FAC posed the following question, “Why 
do students leave the institution?” that goes beyond the currently available data in the 
HS2CSR. 

Appendix: Detailed Summary Information by Campus section. Several of the 
respondents repeated the need to include all students’ data rather than limiting when 
the count is less than 10 in order for the data to be meaningful and helpful. Specific 
requests for additional comparisons with similar schools, with 2-year versus 4-year 
schools, and with enrolled students at the “home” institution versus other institutions 
were included. On the other hand, concern about school-to-school comparisons was 
expressed by one respondent: “To compare students in my district who are receiving 
below the state average in per pupil funding to students in districts where the per-pupil 
funding is 2 or 3 times the state average is not comparing apples to apples.” 

Illinois Custom Addendum Charts and Tables. Only a few respondents provided 
feedback for this section. Some found the data helpful and would like additional 
comparison data. Others found the data uninformative and indicated that they have 
access to other sources (e.g., teachers) for similar information. A couple of specific 
suggestions included providing disaggregated data to identify specific areas of deficiency, 
adding cumulative percentages of the totals, and identifying when data are statistically 
significantly different. 
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Conclusions and Recommendations 


Results from the survey and qualitative data affirmed the usefulness of the HS2CSR to 
facilitate improved student achievement by fostering communication and collaboration 
between secondary and postsecondary educators and policymakers. Respondents indicated 
they typically use the HS2CSR to monitor student success. More collaborative activity 
utilizing the report occurred within the K-12 sector and occurred within institutions, 
rather than cross-institutions. Results also identified limitations with the dissemination 
strategy and concerns about the comprehensiveness of the data. Respondents’ suggestions 
to improve the report include increasing the ease of use, increasing the representativeness 
of the data, and providing additional data elements. Recommendations based on the 
results and respondents’ suggestions are provided below. 

Improve dissemination. Survey results indicated that the primary dissemination of 
the HS2CSR to K-12 principals and superintendents, as well as to community college 
and university presidents, was limited, as well as the secondary dissemination to those 
beyond this initial group. Lack of awareness of the report may have contributed to the 
distribution efforts. Without an effective distribution of the report, use of the data and 
collaborations will be limited. Given the typical protocols within education institutions, 
it will be necessary to continue to first communicate and distribute the report to the 
senior level administrators in school districts, community colleges, and universities, as 
was done with the Year 1 and Year 2 reports. In order to reach educators beyond the 
initial distribution list, the IBHE, ICCB, and ISBE should continue to encourage senior 
administrators to distribute the report widely. Additionally, these state agencies could 
target specific subgroups that are likely to be involved in school improvement planning, 
curriculum alignment with the Common Core State Standards, and cross-institution 
collaborations, as well as other activities focusing on improving student achievement. 
Examples of these groups include regional superintendents, National Board Certified 
Teachers, high school and postsecondary counselors, transfer coordinators, institutional 
researchers, and secondary and postsecondary faculty advisory councils. Many of these 
groups have state-level professional organizations that can facilitate announcing the 
availability of and access to the report. Results from our survey indicate direct email to 
all stakeholders would be the most effective strategy. Additional avenues could include 
notifications on the IBHE Friday Memo, the ISBE superintendent weekly message, and 
professional organizations’ newsletters. 

Increase the use of the report. The majority of those who did receive the HS2CSR are 
using it and Ending it useful. They see the report as another data tool and are using it 
in their collaboration activities. Results from the evaluation, however, indicated some 
respondents’ concerns about the representativeness of the data inhibit their use of the 
report and collaborative activities. Respondents provided several suggestions to reduce 
their concerns: 1) improve the comprehensiveness of the report by including data from 
out-of-state and private institutions; 2) include all data even if the count is less than 10 
so as to not limit the small high schools’ use of the report; and, 3) rely less on self-report 
data. Some of these concerns will be addressed when the new Illinois Longitudinal Data 
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System comes online. Until then, it will be helpful to acknowledge these limitations 
in the report materials. Furthermore, the HS2CSR competes with many other data 
reports for educators’ time and attention. Enhancing the value of the report by providing 
additional data (e.g., track data over time, include comparison data) will likely increase 
utilization of the report. 

Increase ease of use. Results from the evaluation also indicated respondents’ lack of 
time and resources reduce use of the HS2CSR and collaborative activities. Improving 
the ease in which to use the data will facilitate more productive and efficient use of 
time, thus, increasing educators’ likelihood of using the report. Several avenues might 
be pursued to make the data more accessible. First, providing a brief executive summary 
(2-3 pages) will expedite interpreting the data and disseminating the results to a wider 
audience (e.g., school boards, faculty, parents). Second, providing the data in an 
online relational database will allow users to drill down to more detailed data and to 
create custom reports for their specific questions. It will also allow them to more easily 
incorporate the HS2CSR data with their local data systems. Providing the data online 
in an interactive report to view results easily will facilitate use of the HS2CSR. Illinois’ 
recently implemented 5Essentials Survey (http://uchicagoimpact.org/5essentials) 5 to 
measure school climate and inform school improvement activities, provides an excellent 
example of an online, interactive data resource that allows stakeholders to view high level 
information and detailed data results to inform their decisions. Proving descriptions of 
the data sources and definitions will improve clarity and allow users to focus on working 
with the data rather than expending time answering questions about meaning and 
sources of data elements. Regarding time constraints specifically at the high school level, 
setting aside time for analyzing and interpreting data is a prerequisite for data-informed 
decisions (Means, Padilla, Gallaher, 2010; Cosner, 2012). Cosner’s (2012) research on 
instructional reforms also emphasizes the importance of school leadership to set aside 
time for collaborative activities, as well as monitor and make necessary adjustments so 
that the collaborative activities continue to be productive. Another avenue suggested 
by research involves providing technical support for data interpretation through the 
use of data coaches, although Means, et al. (2010) found that this is the least common 
support provided. For long-term impact, policies directed toward pre-service teacher 
education programs to improve instruction on data use for soon-to-be teachers should 
be considered (Means, Padilla, DeBarger & Bakia, 2009). 

Increase collaboration. Encouraging collaborations between secondary and 
postsecondary institutions is one of the primary goals of the HS2CSR. Many respondents 
reported being engaged in collaboration activities, typically within their institutions. 
And while some individuals reported the HS2CSR inhibited collaboration due to data 
issues, many indicated the FIS2CSR facilitated their collaborative activities. When cross- 
institution collaborations do occur, they usually are carried out between the K- 1 2 and 
community colleges, most likely due to the higher rate of developmental education at 
the community college level, rather than at the 4-year institutions. 


5 The 5Essentials was developed by the University of Chicago Consortium on School Research at the 
University of Chicago Urban Education Institute, in partnership with Chicago Public Schools. 
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A desire for more conversations across the education sectors was expressed by many 
survey respondents and focus group participants. In addition to the willingness to work 
together, other conditions are also in place to facilitate collaborations, namely, existing 
relationships among individuals/institutions and the availability of the data provided 
in the HS2CSR (as well as other sources) to inform discussions. IBHE, ICCB, and 
ISBE should build on these current collaborative relationships and encourage new 
collaborations, particularly cross-institution collaborations. 

Several past and current state initiatives (e.g., College and Career Readiness Grants, 
Alliance for College Readiness, Pathways to Results) have had success with developing 
collaborations across education sectors. In general, these initiatives provided funding, 
training, and support to selected community colleges and their partnering high schools 
to target college readiness through various activities, including discussions on curriculum 
alignment and creating interventions to reduce the number of students needing 
developmental education. The HS2CSR was utilized in some of these collaborations 
primarily as a “conversation starter” to help identify potential areas for improvement 
and as an additional data resource that provided a higher level of information that 
facilitated initial conversations between the sectors. For these and future initiatives, 
IBHE, ICCB, and ISBE could position the HS2CSR as a valuable data resource to 
inform the collaborative activities. Identifying data needs for these types of collaborations 
could also inform development and modifications of future reports. 

IBHE, ICCB, and ISBE might consider hosting regional workshops similar to those held 
in Fall 2011 after the release of the Year 1 report to offer training with data interpretation 
and utilization in order to enhance awareness of the report and to facilitate use of the 
HS2CSR and cross-institution collaborations. Online webinars could also be developed 
to provide “virtual data coaches” to support interpretation and use of the data. These 
webinars would also allow flexibility to be accessed at the time and location convenient 
to the participants. The workshops and webinars could engage current high-end users 
to share their strategies and experiences in collaborating and utilizing the HS2CSR, 
particularly around alignment of the Common Core State Standards. These would meet 
the requests of many respondents who expressed interest in learning how other schools 
are utilizing the HS2CSR. 

Multi-layer collaborations, such as the cross-institution collaboration discussed here, 
may have additional “complexities” that make collaborative data practices particularly 
challenging (Nelson, 2008, p. 577). The historical “finger pointing” among education 
sectors may create barriers to achieving successful collaborations. However, the increased 
focus on many statewide education reform initiatives (e.g., Common Core State Standards 
and Illinois P-20 Council goal of 60% with a degree or credential by 2025) provides a 
platform to engage in collaborations. Existing and new cross-institutional collaborations 
developed through collegial trust (Cosner, 2009), plus the use of the HS2CSR, as well 
as many other tools, will support efforts to improve student achievement in Illinois. 
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To conclude, experiences with the first two HS2CSR releases provided opportunities to 
learn what went well and where improvements can be implemented. Results from this 
evaluation show that although the report is not as widely disseminated as hoped, many 
who have received it are using it and finding it helpful in their collaborations within 
their institution and some are utilizing it across institutions. Development of future 
reports should focus on improving the representativeness of the data and increasing ease 
of use. These enhancements, along with improving report dissemination and providing 
training to more fully utilize the report would likely result in expanded use to foster 
communication and collaboration among secondary and postsecondary educators to 
improve student achievement. 
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Appendix A 


High School to College Success Report 
Online Survey 



Welcome! Thank you for your efforts in helping us to understand how you use the Illinois High School to College 
Success Report (HS2CS). Our organizations, the Illinois Education Research Council (IERC) and the Applied 
Research Consultants (ARC), have been asked to evaluate the dissemination and usefulness of the HS2CS 
reports as well as your collaborative efforts within and across institutions. Beginning in July 201 1 , IBHE, ICCB, 
and ISBE, in conjunction with ACT, released a new format of the report containing data on scores and grades 
for three consecutive school years. The goal of the report is to encourage data-driven decision making and 
collaboration between secondary and post-secondary stakeholders through the use of college retention rates, 
college grade point averages, and high school preparation for college statistics. Your feedback is crucial to 
examining the usefulness of the report! With this in mind we would greatly appreciate your help and we ask that 
you complete the following questionnaire about your experiences with the report. In the survey we ask everyone 
to complete questions pertaining to: your overall opinion of each section of the report, the dissemination of 
the report, usage and usefulness of the report, suggested data elements to add or delete, your collaborative 
activities involving the report. We estimate the time to complete the survey to be 1 0-20 minutes, depending on 
your responses. Thank you in advance for your assistance. It is greatly appreciated. The deadline to complete 
the survey is: Friday, March 29, 2013 


T SOUTHERN ILLINOIS UNIVERSITY 

Illinois Education Research Council oA U APPLIED RESEARCH 

CARBONDALE CONSULTANTS 

Informed Consent This Informed Consent needs to be read and electronically signed by you if you wish to 
participate in this research study. This online survey was designed to gather information regarding your knowledge 
and opinion regarding the High School to College Success report for the state of Illinois. Additionally, this online 
survey will seek information regarding potential changes to the report and your collaborative efforts in using 
the report. You represent a sample of the population being researched as you are the main audience for the 
report. Participation is voluntary. You may choose not to participate in this online survey. When participating 
in this online survey, your responses will be confidential. We will maintain your contact information only for 
reminder purposes or if we have questions regarding your responses. No names or identification numbers will 
be connected to the analyses or reports. Data will be reported in aggregate form. Completion and return of 
this survey indicate voluntary consent to participate in this study. If you have any questions about this study, 
contact: Brenda K. Klostermann, PhD Associate Director, Illinois Education Research Council (IERC) Southern 
Illinois University Edwardsville Email : breklos@siue.edu Phone : (61 8) 650-2840 Applied Research Consultants 
(ARC) Department of Psychology Southern Illinois University Carbondale Email : arcmail@siu.edu Phone : 
(618) 453-3536 

Instructions: Please click the “I Consent “ button below to continue with the survey. If you do not wish to participate 
please exit the browser. 

This study has been reviewed and approved by the SIUC Human Subjects Committee and the SIUE Institutional 
Review Board. Questions concerning your rights as a participant in this research may be addressed to the 
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Committee Chairperson, Office of Sponsored Projects Administration, Southern Illinois University, Carbondale, 
IL 62901- 4709. Phone: (618) 453-4533. E-mailsiuhsc@siu.edu 

Instructions: We are interested in your experiences with the Illinois High School to College Success Report. 
Please answer the following questions as accurately as possible. This is not a test and therefore there are no 
“correct “ answers. Your responses are simply an opportunity for us to gain more insight on how you use this 
report in your education planning or evaluation practices; what problems or opportunities you have experienced 
with this report. For your convenience we have provided the link to State Reports. Please feel free to cross- 
reference your report while completing this questionnaire. (Click on your category of institution.) Four-year 
public institutions Community Colleges High Schools - Each school’s report is posted on the profile page of the 
high school on the Illinois Interactive Report Card. Select a high school of your choice to use as an example. 
Important: If you need to go back to a previous page you must use the back button provided within the survey. 
Do not use your internet browser back button. You may exit the survey and return at a later time. Before you 
exit, complete the question(s) on the current screen. All responses will be lost on the incompleted screen. To 
return, use the survey link provided in the email we sent you. 


Q1 In which region of the state do you work? 

O Chicago 

O Northeast (not including Chicago) 

O Northwest 
O East Central 
O West Central 
O Southeast 
O Southwest 

Q2 What is your title/role at your institution? 

O Principal 

O Superintendent (district or ROE) 

O 2-year college Administrator 
O 4-year college Administrator 

O Not Listed, (please provide) 

Q3 Please indicate the number of years of experience in your current position. 

Q4 Indicate the number of years employed in education. 

Instructions: The High School to College Success reports are sent to public education institutions through U.S. 
mail and are posted on IBHE, ICCB, ISBE, and Illinois Interactive Report Card (IIRC) websites. The reports may 
have been circulated within institutions. The first report was delivered in Summer 2011 and the second report 
in Winter 2013. Please answer the following questions in regards to your experience with the report(s). The 
2006-2008 Freshmen Report - Delivered in Summer 2011 - Referred to as the Year 1 report. The 2007-2009 
Freshmen Report - Delivered in Winter 2013 - Referred to as the Year 2 report. 
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Q5 Which report(s) did you receive? 

O I received both reports. 

O I only received the Year 1 report. 

O I only received the Year 2 report. 

O I did not receive either report. 

Q6 From whom did you receive the Year 1 report or how did you access the Year 1 report? (check all that apply) 

□ Supervisor/administrator 

□ Colleague 

□ Direct mailing of institutional CD 

□ IBHE website 

□ ISBE website 

□ ICCB website 

□ Illinois Interactive Report Card (IIRC) 

□ Other 

Q7 How would you prefer to be notified when the Illinois High School to College Success Report is available? 
(check all that apply) 

□ Email notification 

□ Your professional organization 

□ IBHE Friday Memo 

□ ISBE Superintendent’s weekly message 

□ School newsletter 

□ ROE notification 

□ Other (Please list): 

Q8 How would you rate the effectiveness of the Year 1 report’s dissemination? 

O I received the report in a timely manner 

O I received the report late, but was still able to utilize it 

O I received the report too late to use it in my education practice 

Q9 To what extent did you read the Year 1 report? 

O Not at all 
O To some extent 
O To a great extent 
O I have read the full report 


41 


Evaluation of the Illinois High School to College Success Report 


Q10 From whom did you receive the Year 2 report or how did you access the Year 2 report? (check all that apply) 

□ Supervisor/administrator 

□ Colleague 

□ Direct mailing of institutional CD 

□ IBHE website 

□ ISBE website 

□ ICCB website 

□ Illinois Interactive Report Card (IIRC) 

□ Other 

Q11 How would you prefer to be notified when the Illinois High School to College Success Report is available? 
(check all that apply) 

□ Email notification 

□ Your professional organization 

□ IBHE Friday Memo 

□ ISBE Superintendent’s weekly message 

□ School newsletter 

□ ROE notification 

□ Other (Please list): 

Q12 How would you rate the effectiveness of the Year 2 report’s dissemination? 

O I received the report in a timely manner 

O I received the report late, but was still able to utilize it 

O I received the report too late to use it in my education practice 

Q13 To what extent did you read the Year 2 report? 

O Not at all 
O To some extent 
O To a great extent 
O I have read the full report 

FOR THOSE WHO DID NOT RECEIVE REPORTS 

Q14 Were you aware of the Illinois High School to College Success Report before receiving this survey? 

O Yes 
O No 
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Q1 5 Before you leave, we are interested in other resources you may be using to collaborate with those within or outside 
your institution. Therefore, what other data/informational resources are you using in your collaboration efforts to 
improve student success (e.g., curriculum alignment, recruitment, access, retention, funding, implementation 
of the Common Core State Standards)? 

Thank you for completing this survey! The information you provided will help achieve a greater understanding 
about the Illinois High School to College Success Report. Your feedback will also help us to better understand 
how you are using the report. Again, thank you for your feedback! Please be sure to click the submit button 
below to save your answers. 


FOR THOSE WHO RECEIVED REPORTS 

Instructions: You have indicated that you have received and/or read the report. In the following questions you 
will be asked about your use of the report(s). We are interested in whether you have used the report(s) and 
specifically which sections of the report you have used. 

Q16 Have you utilized the High School to College Success Report(s)? 

O Yes 
O No 

Q17 Please indicate why you do not utilize the report(s). (Check all that apply) 

□ Too much information 

□ Data are not reliable 

□ Not enough information 

□ Better resources, with similar information, exist 

□ The report is confusing 

□ The data is outdated 

□ The data is not relevant to my work 

□ I have not had time to review, but plan to utilize, the report 

□ Other 

Q18 How are you using the report(s)? (check all that apply) 

□ Making program changes 

□ Collaborating within my institution 

□ Collaborating outside my institution 

□ Making curriculum/course changes 

□ Monitoring student success 

□ Making changes in advisement 

□ Other 
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Q19 To what extent do you utilize the High School to College Success Report? 

O Very Rarely 
O Rarely 
O Occasionally 
O Frequently 
O Very frequently 

Q20 I consider the High School to College Success Report to be 

O Not at all useful 
O Slightly useful 
O Moderately useful 
O Very useful 

Q21 What data elements could be included to make the report more useful? 


Q22 Which data elements are not useful and could be removed to make the report more concise? 


Instructions: The next several questions will ask you about your use of each section of the report. Please indicate 
which section(s) of the report you utilize. Page numbers of each section have been provided for ease-of-reference. 
For your convenience we have provided the link to State Reports. Please feel free to cross-reference your 
report while completing this questionnaire. (Click on your category of institution.) Four-year public institutions 
Community Colleges High Schools - Each school’s report is posted on the profile page of the high school on 
the Illinois Interactive Report Card. Select a high school of your choice to use as an example. 

Instructions: Please indicate whether you use the following section of the report. Charts included in the High 
School Preparation and Success section: 

Chart 1: High School and First Year College GPA for Local and Statewide Students - All Students and 
Those Assigned to Developmental Courses 

Chart 2: Average First Year College GPA for Students Who Did/Did Not Earn ACT College Readiness 
Benchmark Scores Across Test Subjects 

Chart 3: Percent ‘Below’ and ‘At or Above’ a First Year College GPA of 2.50 by Mathematics Course 
Sequence Patterns Studied in High School 

Chart 4: Percent ‘Below’ and ‘At or Above’ a First Year College GPA of 2.50 by Science Course Sequence 
Patterns Studied in High School 

Chart 5: Local and Statewide First Year College GPAs by ACT College Readiness Standards Score Ranges 

Q23 Have you used the High School Preparation and Success (p. 1) section? 

O Yes 
O No 
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Q24 Overall the High School Preparation and Success section (containing charts 1-5) is useful. 

O Strongly Disagree 
O Disagree 

O Neither Agree nor Disagree 
O Agree 
O Strongly Agree 

Q25 Please share some examples on how to make the High School Preparation and Success Section more useful. 


Instructions: Please indicate whether you use the following section of the report. Charts included in the College 
Readiness and Success section: 

Chart 6: First Year College GPA by ACT College Readiness Standards Score Ranges and Test Subjects 

Chart 7a: Local and Statewide ACT Composite Test Scores for All Students and for Students Taking 
Developmental Courses by Core Course-Taking 

Chart 7b: Local and Statewide First Year College GPAfor All Students and for Students Taking Developmental 
Courses by Core Course-Taking 

Chart 8: Local and Statewide Average ACT Scores for Students Assigned to Developmental Coursework 
in College Across Test Subjects 

Q26 Have you used the College Readiness and Success (p. 6) section? 

O Yes 
O No 

Q27 Overall the College Readiness and Success section (containing charts 6-8) is useful. 

O Strongly Disagree 
O Disagree 

O Neither Agree nor Disagree 
O Agree 
O Strongly Agree 

Q28 Please share some examples on how to make the College Readiness and Success Section (containing charts 
6-8) more useful. 
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Instructions: Please indicate whether you use the following section of the report. Charts included in the College 
Success and Persistence section: 

Chart 9: Local and Statewide Students Enrolled in Public Postsecondary Institutions who Did/Did Not 
Return to Same Campus in Year 2 -First Year College GPA 

Chart 10: Local and Statewide Students Enrolled In Public Postsecondary Institutions Who Did/Did Not 
Return To Same Campus In Year 2 - ACT Composite Score And High School GPA 

Q29 Have you used the College Success and Persistence (Charts 9 and 10; p. 10) section? 

O Yes 
O No 


Q30 Overall the College Success and Persistence section (containing charts 9 and 10) is useful. 

O Strongly Disagree 
O Disagree 

O Neither Agree nor Disagree 
O Agree 
O Strongly Agree 


Q31 Please share some examples on how to make the College Success and Persistence Section (containing charts 
9 and 1 0) more useful. 

Instructions: Please indicate whether you use the following section of the report. Tables included in the Appendix: 
Detailed Summary Information by Campus section of the report: 

Table 1 : Summary Statistics for Your ACT-tested Students Compared to All Enrolled ACT-tested Students 
Statewide 

Table 2: Summary Statistics for Your ACT-tested Students Who Did/Did Not Take Core Coursework 

Table 3: Average First Year GPA and Hours Completed for ACT-tested Students by College Readiness 
Benchmark Scores 

Table 4: First Year College GPA by Mathematics Course Patterns Taken by Your ACT-tested Students 

Table 5: Average First Year College GPA by Science Course Patterns Taken by Your ACT-tested Students 

Table 6: Average First Year GPA for Your ACT-tested Students by ACT College Readiness Score Ranges 

Table 7: Summary Statistics for Your ACT-tested Students Who Were Identified as Needing Developmental 
Coursework 

Table 8: Summary Statistics for Your ACT-tested Students Enrolled in Public Postsecondary Institutions 
Who Die/Did Not Return to the Same Campus in Year 2 
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Q32 Have you used the Appendix: Detailed Summary Information by Campus (Appendix Tables 1 -8; p. 1 3) section? 

O Yes 
O No 


Q33 Overall the Appendix: Detailed Summary Information by Campus section (containing tables 1 - 8) is useful. 

O Strongly Disagree 
O Disagree 

O Neither Agree nor Disagree 
O Agree 
O Strongly Agree 


Q34 Please share some examples on howto make the Appendix: Detailed Summary Information by Campus section 
(containing tables 1 - 8) more useful. 


Instructions: Please indicate whether you use the following section of the report. Charts included in the Illinois 
Custom Addendum section of the report: 


Addendum Chart 1 : Local and Statewide Average ACT Scores for Students Assigned to Developmental 

Coursework in Language Arts Across Test Subjects 

Addendum Table 1: Average ACT Scores for Your Students Who Were Assigned to Developmental 

Coursework in Language Arts 

Addendum Chart 2: Local and Statewide Average ACT Scores for Students Assigned to Developmental 

Coursework in Mathematics Across Test Subjects Addendum Table 2: Average ACT 
Scores for Your Students Who Were Assigned to Developmental Coursework in 
Mathematics 


Addendum Chart 3a: Local and Statewide Students Enrolled in Public Postsecondary Institutions Who Did/ 
Did Not Return to Any Public Institution for Year 2 - First Year College GPA 

Addendum Chart 3b: Local and Statewide Students Enrolled in Public Postsecondary Institutions Who Did/ 
Did Not Return to Any Public Institution for Year 2 - ACT Composite Scores and High 
School GPA 


Addendum Table 3: Summary Statistics for Your ACT-tested Students Who Did/Did Not Return to Any 

Public Institution for Year 2 


Q35 Have you used the Illinois Custom Addendum (p. A1 ) section? 

O Yes 
O No 
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Q36 Please indicate your level of agreement with the following statements. 



Strongly Disagree 

Disagree 

Neither Agree nor 
Disagree 

Agree 

Strongly Agree 

Overall the 
Illinois Custom 
Addendum Charts 
are useful. 






Overall the 
Illinois Custom 
Addendum Tables 
are useful. 







Q37 Please share some examples on how to make the Illinois Custom Addendum Charts more useful. 

Q38 Please share some examples on how to make the Illinois Custom Addendum Tables more useful. 

Instructions: Rank-order the sections of the report in terms of their usefulness. Assign each section a number 
(1-5) that represents how you rank each on usefulness, with 1 being the most useful and 5 being the least 
useful. 

Q39 Which sections are the most useful? 

□ High School Preparation and Success (p. 1) 

□ College Readiness and Success (p. 6) 

□ College Success and Persistence (Charts 9 and 10) (p. 10) 

□ Detailed Summary Information by Campus (Appendix: Tables 1 - 8; p. 13) 

□ Illinois Custom Addendum (p. A1) 

Instructions: Some institutions use the data in the High School to College Success reports to collaborate among 
staff and faculty within their own institutions and with other institutions on ways to improve student success 
throughout the pipeline of high school to college graduation. In this section we’re interested in your experiences 
with collaboration and how this report has been used in (or could be improved for use in) these collaborations. 
Collaboration efforts could involve improving curriculum alignment, recruitment, access, retention, funding, 
implementation of the Common Core State Standards, among other things. 

Q40 To what extent have you collaborated using the Illinois High School to College Success Report? 

O Never 
O Seldom 
O Sometimes 
O Often 
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Q41 You have indicated that you are not collaborating with other institutions using the Illinois High School to College 
Success Report. Please indicate below any possible reasons as to why you are not collaborating, (check all 
that apply) 

□ The report does not contain the information I need to enhance my collaborations. 

□ The report is too difficult to understand. 

□ The data in this report are not relevant to the type of collaborations we do with other institutions. 

□ A lack of time prevents me from reading this report in preparation for my meetings with other 
institutions. 

□ Other institutions, in which I could collaborate, are not using the report. 

□ Other Reason 


Q42 To what extent have you collaborated with individuals inside your institution to improve student success? 

O Never 
O Rarely 
O Sometimes 
O Often 

Q43 The Illinois High School to College Success Report has been helpful in collaboration among the staff inside 
my institution. 

O Strongly Disagree 
O Disagree 

O Neither Agree nor Disagree 
O Agree 
O Strongly Agree 

Q44 To what extent have you collaborated with individuals outside your institution to improve student success? 

O Never 
O Rarely 
O Sometimes 
O Often 

Q45 The Illinois High School to College Success Report has been helpful in collaboration with individuals outside 
my institution. 

O Strongly Disagree 
O Disagree 

O Neither Agree nor Disagree 
O Agree 
O Strongly Agree 
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Q46 What barriers have you experienced in your attempts to collaborate with others to improve student success? 

Q47 What has facilitated collaboration with others to improve student success? 

Q48 To what extent do you agree that a summarizing brief (e.g. 2-3 pages) of the High School to College Success 
Report for each institution would be useful for collaboration purposes? 

O Strongly Disagree 

O Disagree 

O Neither Agree nor Disagree 

O Agree 

O Strongly Agree 

Q49 How has the HS2CSR changed (or how do you anticipate the HS2CSR will change) your collaboration activities 
to improve student success (e.g., curriculum alignment, recruitment, access, retention, funding, implementation 
of the Common Core State Standards)? 

Q50 What other data/informational resources are you using in your collaboration efforts to improve student success? 

Q51 To what extent does the HS2CSR add to your current collaboration activities to improve student success? 

O None 

O Some 

O Quite a Bit 

O An Extreme Amount 

Instructions: We are interested in learning more about your collaborations. Below we have provided space for 
you to describe examples of two of your collaborative efforts in which you utilized the High School to College 
Success Report. More space is provided if you would like to include additional information. Please include 
the position title of the individual with whom you collaborated and the goal of the collaboration. For example, 
a high school English teacher meeting with a community college English 101 instructor to discuss curriculum 
alignment. 

Q52 Please describe your first example of collaborating with others utilizing the High School to College Success 
Report. 

Q53 What sector(s) is your first collaborator in? (check all that apply) 

□ High School or District 

□ ROE 

□ Community College 

□ 4-year university 
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Q54 Please describe your second example of collaborating with others utilizing the High School to College Success 
Report. 

Q55 What sector(s) is your second collaborator in? (check all that apply) 

□ High School or District 

□ ROE 

□ Community College 

□ 4-year university 

Q56 Would you like to provide us with more collaboration examples? 

O Yes, I have more collaboration examples to provide. 

O No, I do not have more examples. 

Instructions: You have indicated that you would like to provide more collaboration examples. Below is space 
to provide up to four more examples. 


Q57 Please describe your third example of collaborating with others utilizing the High School to College Success 
Report. 

Q58 What sector(s) is your third collaborator in? (check all that apply) 

□ High School or District 

□ ROE 

□ Community College 

□ 4-year university 

Q59 Please describe your fourth example of collaborating with others utilizing the High School to College Success 
Report. 

Q60 What sector(s) is your fourth collaborator in? (check all that apply) 

□ High School or District 

□ ROE 

□ Community College 

□ 4-year university 

Q61 Please describe your fifth example of collaborating with others utilizing the High School to College Success 
Report. 
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Q62 What sector(s) is your fifth collaborator in? (check all that apply) 

□ High School or District 

□ ROE 

□ Community College 

□ 4-year university 

Q63 Do you have any suggestions for who else should receive the report? 

Q64 Is there anything else you would like to add to improve the dissemination or usefulness of the Illinois High 
School to College Success Report? 
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Appendix B 

High School to College Success Report Evaluation 
Focus Group Questioning Route 

Introduction: 

• Please let us begin with a brief introduction, including your name, institution, and how you received the report. 

For our first question: 

• Please describe how you (or your staff) have utilized the report. 

■ Probe: Can you describe one or two examples of your experiences with the report in relation to improving 
student success, achievement, retention, etc.? 

■ Probe: In what way does the report change your institution’s practices? 


Key Questions (Collaboration): 

• Can you describesomeofthecollaborationsyou oryourstaffhavehadusingthe Illinois High School to College Success Report? 
(If they have not collaborated, ask how they could envision using the report in collaboration activities .) 

■ Probe: Just to clarify was that an internal collaboration? 

■ Probe: Just to clarify was that an external collaboration? 

■ Probe: What was the purpose of the collaboration? 

■ Probe: Who were your collaboration partners? 

■ Probe: How often do you meet to collaborate? Is this ongoing, one-time? 

♦ If you only met one time, would it have been beneficial to meet more; or do you feel you accomplished 
your goals? 

• How have your collaboration activities changed since the introduction of the High School to College Success 
Report? Specifically, compare your collaboration efforts prior to your use of this report. 

• What features of the report have been beneficial in facilitating collaboration with others to improve student success? 

• What barriers have you experienced in your attempts to collaborate with others to improve student success while 
using this report (HS2CSR)? 


Key Questions (Suggestions to Improve Report): 

• What data in the current report are most helpful to you, either within your institution or in your collaborations 
with other institutions? 

■ Probe: Are there specific charts or tables you find most helpful? 

■ Probe: Can you tell us why they are helpful? 

• What suggestions do you have to improve the report? 

■ Probe: Specific data elements 

■ Probe: Formatting 

■ Probe: Dissemination 
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• What features that are not currently in the report would help you foster greater collaboration? 

■ Probe: Across institutions? 

■ Probe: Within institution? 

• How useful would an executive summary of the report be to help interpret the data or to help with collaborations? 

• Aside from the HS2CSR, what other informational resources are you using to improve student success? 

■ Probe: Other sources of data 

■ Probe: Other sources to help with collaboration 

• How do these additional resources improve the quality of your collaboration in ways that the HS2CSR does not? 

Conclusion: 

• Overall, how effective do you feel the report is in helping you improve student success? 

• Finally, we would like to go around the room and have everyone indicate the most important point that you would 
like the agencies to hear about your experiences with the report. 

Optional Question Bank (depending on the type of meeting) 

1 . If you have colleagues who do not use the report, what barriers do you think prevent them from making use of 
it? 

2. If training in the use of the report were available, would you attend (either online or in-person) and what topics 
of training would you find most helpful? 

■ Probe: If yes, what would be your preferred option in which to receive training? (face-to-face, webinar, 
handouts, presentations at professional association meetings, district meetings, regional meetings, or other) 
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Appendix C 

Methodology for the Focus Groups and Qualitative Analysis 


After conducting the survey portion of the project, a 
researcher from the Illinois Education Research Council 
and researchers from Applied Research Consultants of 
Southern Illinois University, worked together to develop 
questions based on survey items that required more 
in-depth feedback, as well as follow-up questions. A 
preliminary focus group based survey was sent through 
email to those that were on the email list for the initial 
survey. This survey asked whether or not the individual 
would be interested in participating in a focus group for 
the purpose of providing in-depth feedback and if so what 
medium would they prefer (online, telephone, or face- 
to-face). Once the responses to the focus group survey 
were received, the individuals who indicated interest in 
participating in a focus group, were contacted through 
email to set up times for their participation. 

The focus groups were conducted from May to May 2013 
and through three different mediums; online through 
Go-to-Meeting (an online meeting software) conference 
calls, and face-to-face. In addition, researchers attended 
three separate meetings in March and May 2013 with 
postsecondary senior administers and faculty to request 
feedback on the focus group questions. The meetings 
included: 1) Chief Academic Officers (CAOs) of Illinois 
community colleges held at Richland Community College, 
2) CAOs of 4-year and community colleges held at IBHE, 
and 3) IBEIE Faculty Advisory Council (FAC) which 
includes faculty from both 2-year and 4-year institutions 
held at Monmouth College. It is important to note that in 
order to better accommodate the participants, locations for 
face-to-face focus groups were chosen based on the locations 
of the participants. A few central locations were chosen with 
the aim at preventing participant travel from exceeding 
60 miles one-way. Focus groups were conducted by a 
researcher from the Illinois Education Research Council 
and a researcher from Applied Research Consultants of 
Southern Illinois University. There were a total of 8 focus 
groups conducted, 3 face-to-face, 4 telephone, and 1 online, 
in which a total of 20 individuals participated. All of the 
focus groups were audio recorded, as well as the meetings 
with the 2-year and 4-year of CAOs and the IBHE FAC. 
Thus, the qualitative analysis included the 10 recordings 
plus researcher’s notes taken at the IBHE FAC meeting. 

The first step of the qualitative analysis was to transcribe the 
recordings from each focus group. Researchers from Applied 


Research Consultants of Southern Illinois University did 
the transcriptions. The second step required three separate 
judges to develop categories that emerged from participant 
responses. The third step involved an analysis of the 
frequency of responses across those categories across each 
question. The development team consisted of three judges, 
two auditors, and a research consultant. All three judges 
were advanced graduate psychology students; they were all 
females. The auditors were Ph.D. level psychologists, who 
were both females. The research consultant was a female 
Ph.D. level Industrial and Organizational Psychologist. 

There were cases where individual responses were 
commonly assigned to multiple codes as some responses 
contained multiple concepts. Then, the judges met with 
the research consultant to generate an initial coding system 
which consisted of a list of all codes that emerged from the 
responses. The judges discussed the codes and arrived at 
consensual agreement about the most appropriate codes 
that captured the essences of the data. The coding scheme 
served as a conceptual framework by which to organize 
the data. 

After the coding scheme was developed, analysis of all 
of the data was initiated. Each judge was given all of the 
responses to each of the questions. Each judge coded each 
response assigning codes to individual responses. The judges 
collapsed, revised, or discarded their codes into overarching 
categories. The categories represented superordinate 
themes that cut across the codes developed. These findings 
were then presented to the auditors. The auditors noted 
inconsistencies (e.g., same response being assigned different 
codes) and aspects of the coding system that lacked clarity 
(e.g., meaning of a particular code). 

The judges then began refining the categories again, making 
consensual decisions in response to the auditor’s feedback, 
and making revisions to the codes assigned and coding 
scheme. The auditors assisted in reaching consensus with 
the judges that data saturation had been achieved after 
all the responses were analyzed. Through the process of 
developing the categories, the development team was able 
to ensure that no new themes emerged from the data and 
that the categories were representative of the experiences 
of all the participants in the study. 


55 


Evaluation of the Illinois High School to College Success Report 


Appendix D 


Examples of Collaborations 


Collaboration #1 

• We exchanged data with a community college. We have tried to have more collaboration, but they are not very easy 
to work with. Their unions make it difficult and the turnover in their administration has not helped. 

• High school faculty — the focus from their perspective became the differences in our local grades and grades students 
receive in post-secondary work. 

• Attended a regional meeting on the topic 

• I did not personally take part; however, the Associate Dean of Mathematics and several math faculty have been 
meeting with the faculty from several district high schools to better align the curriculum. Their goal is to close the 
expectation gap between high school math and the college placement exam. 

• Joint in-services every fall of HS and CC faculty 

• I have worked with our high school departments to try and prepare our students for success after high school. We 
have encouraged as many students as possible to take a 4th year of mathematics. Those who do are very likely to 
test high enough on the asset test to allow them to take college level math classes. Those students who choose not to 
take a 4th year of high school math often are placed in college remedial math. Some of these students were decent 
math students their first three years of high school, but the year away from math caused them to slip significantly. 
If you don’t practice math regularly, you are likely to forget it. 

• Meetings with High School English and Math instructors - curriculum alignment and Dual Credit Quality Act 
and LAI requirements. 

• I attended a workshop 2 years ago when this report was introduced. There were people from HS and Higher Ed 
at the meeting and we talked together about the information. 

• Preventing colleagues from thinking that it is representative of their school. It is useful at the individual institution 
level, but even then, serious caveats apply (e.g., too many unaccountably missing students). 

• We shared the data with our high school departments, so that they could see linkages between their instruction, 
ACT scores, and college success rates. 

• HS faculty underwent a “jigsaw” activity of section 1 at an hour early dismissal in-service. 

• Discussing the results to understand what they actually mean. 

• Teacher meetings and workshops. 

• Please fix the science benchmark. It will help us utilize the power of this report. 

• Our collaboration has been mainly administratively and with our part-time curriculum person. Very little at this 
point with department leaders. We plan to use it with our Rising Star school improvement team also. 

• During administrators meeting with building administrators who then took it to data team meetings to look for 
areas of improvement. 

• The district and high school administrative teams began to collaborate; the Superintendent led pipeline discussion 
with the entire K-12 admin team to build the pathway for accelerated progress and increasing expectation from 
K-12. Out of these discussions came staffing changes; co-curricular programs after school and in the summer that 
went beyond at-risk student populations; increased AP training, course offerings, and enrollments. 

• I used the high school to college success report in a presentation to local high schools about College Readiness. 

• We had a meeting with our high school partners to discuss the report. 
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• Internal collaboration within the college 

• Discussion among principals of EPAs progression and how to use the data 

• Using the data in conjunction with our EPAS testing has allowed us to be on the same page with our feeder schools, 
and created more need for ongoing articulation discussions. 

• Board report 

• Our counselors use the information on college persistence to better understand which schools may be right for 
our students. 

• Teachers internally within departments regarding AP and dual credit course offerings. 

• Analysis of data with school staff 

• Please keep in mind that transparent conversations about student preparation and success as relatively new, but are 
now ongoing with all district high schools. During our most recent discussions with district superintendents and with 
high school principals, we have referred to these data as we share our work in improving student learning success. 

• Director of Student Success and Accountability met with the Assistant Superintendent and Superintendent to 
debrief them on the contents of the report. It was concluded that our students’ grades were not being inflated. Our 
students have similar college GPAs to their high school GPAs; the same cannot be said of the state. 

• SIP Team, Common Core Team 

• I worked with our Curriculum Director and Guidance Counselor to evaluate the success of our students after 
high school. 

• We have been working with two local high schools in attempt to increase their ACT Compass scores. Teams from 
each school have looked at transition data for a few years. This report is one more resource that can help advise the 
teams about where to focus their efforts. 

• Public High School Superintendents at a partnership meeting 

• High school English, Math, Science, History, World Language, and Special education teachers meeting together 
to analyze data and design, develop, and implement programs to ensure student success 

• Keven Hansen SWIC discussing math readiness 

• Our dual credit programs appear to be coming to an end or to be substantially weakened. The HSCSR will facilitate 
more dialog between college and high school instructors. 

• Meeting with high school counselors on a regular basis. 

• With the H.S. Principal- explaining the impact of raising rigor level on student achievement in enrichment level 
classes 

• Meetings where scores are reported, shared and discussed 

• In Waukegan, we have joint college and career readiness meetings monthly to increase awareness of 21st Century 
Learning skills and to match students with mentors. Reaching out to students to raise their awareness about college 
and career strategies and or requirements offers our students a win- win opportunity. 

• Guidance counselor 

• Creating a moral imperative in terms of how well we are preparing our students. This is used in combination with 
a great deal of other data points. 

• Useful in Curriculum development committee planning. 

• School Improvement team meetings 

• Talked to local community college. 

• Met with other local curriculum specialists and the local community college to discuss potential gaps in student 
attainment. Looked for root cause. 
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• Tony Capalbo, Director of Partnership for College and Career Success program at McHenry County College and 
I have collaborated to adjust the ACT math score used to identify students for the remedial math program at the 
college. 

• We utilized this information to plan course work through Wabash Valley College for students who plan transfer 
degrees to 4 year universities. 

Collaboration #2 

• High school faculty — connecting the benchmarks with our EPAS growth analysis. 

• Attended a regional meeting on the topic 

• Data sharing of HS students first semesters college performance 

• We have worked with our local junior college to help students be more successful once they leave high school. We 
know the key is prior preparation. We strongly encourage our students to take challenging math, English and science 
courses all four years of high school but too often that is not the case for our struggling students who continue to 
struggle once they are in college. 

• Community College and High School counselors’ meetings to discuss curriculum, gaps and misunderstandings. 

• Student Services personnel analyzed the data on persistence rates to try to target students at risk of starting college 
but not finishing. 

• HS administration shared results with parents at a HS to College forum 

• Discussion of why our students don’t perform well at the community college level compared with other students. 

• School Board Reports and parent/teacher conferences. 

• Sharing the connection between ACT performance and college performance with our own kids. Powerful. 

• The other high school in our district. 

• Sharing with BOE. 

• The high school to college success report was useful for framing discussions between high school and college faculty. 

• Discussion of post-secondary success and how to measure and track student successes 

• The four year university information allows our guidance counselors a dual structure of working with departments 
and teachers as well as with parents and students on four year planning 

• Dissemination and discussion with counselors 

• We review the results with our community college. 

• Dialogue with area community college regarding dual credit success rate. 

• With students and families to encourage enrollment in additional or more rigorous courses, which is an indicator 
of increased college success. 

• During our monthly Student Success Team meetings as we pursue our pilot projects in student placement, course 
learning outcomes, retention, and success. 

• Outside presenters through the ROE 

• We worked with our School Improvement team to discuss the success of our students after high school. 

• We have had a few meetings concerning the common core competencies and efforts to align all of our high schools 
curricula. This report assists us in understanding some of the challenges many schools in our district face. 

• High school computer science and engineering teachers collaborating with local university computer science and 
engineering professors to improve computer science and engineering courses 
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• Paul Stevenson of Downers Grove South HS 

• Working with other community colleges on best practices. 

• Serving on the Steering Committee of our local community college. 

• Department chairmen 

• This has helped us expand our Dual Credit program. 

• Strategic Planning Team 

• Talked to my high school faculty. 

• Met with local curriculum specialists to discuss their successes and opportunities. 

• McHenry County College hosted workshops related to ELA/Math and the Common Core State Standards in 
which various high school teachers and administrators from the county attended. 

Collaboration #3 

• Community college and high school administrators to discuss misalignment, consolidation of services, etc. 

• School Board members 

Collaboration #4 

• Training opportunities for high school and community college educators to improve skills and ultimately address 
the community college district’s areas of concern. 

• Admission counselors at CC 
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Appendix E 

Survey Respondents’ Suggested Improvements 
for Report Sections 

High School Preparation and Success Section 

• Again, put this online in a relational database where custom reports can be generated. 

• For the less than three years of math and science, what are the content areas? What is general science? Rarely do 
high schools run such a vague course, rather, they run biology, chemistry, etc. 

• I am not surprised that students’ college GPA’s drop, but this will be valuable to track over time. It is not too 
valuable to me as a stand-alone piece of data. Again, what classes and majors does this represent? 

• I liked the comparison of our students with an ACT score earned ‘x’ college GPA much more than a correlation or 
comparison between high school GPA and college GPA. If ACT is an aptitude test, then we should be measuring 
our 24s against state 24s to see if our 24s were more prepared. I found that to be a fascinating section — again — 
graduate years must be broken out to maximize impact of the report. 

• Ideally, the charts should compare high schools with similar demographics. 

• Indicate how the data is obtained, (from the high school, self-reported by students, etc) 

• It is a good report as is. 

• It strengthens the community college argument for a mandatory senior math course in high school. 

• Looking at the GPA gap from HS to college started some district discussions 

• More current information. Break each chart out by HS in the college report. 

• More data from all schools (public, private, out of state) 

• Not sure 

• Once courses are aligned to the core competencies this data will be more valuable. At present it is difficult to 

understand at school level because they courses they take have differences (i.e. English 1A, IB, and 1C). 

• Pages 1 and 5 are marginally interesting, the rest can be eliminated. The benchmarks are too inaccurate and invalid 
to be useful, core course info is trivial and too likely to be inaccurate (self-reported), Incidentally, there was a page 
numbering problem on our 2006-2006 report. Since I can’t go back to the earlier part of the survey, I wanted 
to correct what I wrote; it was pages 13, 14 and (to a lesser degree, and ignoring the benchmark info) 20 on the 
2008-2010 report that were useful 

• Put the state average on Chart 2, 3, and 4. 

• Rather than comparing all vs developmental, consider developmental vs non-developmental 

• The ACT/EPAS College Readiness Benchmarks that are cited in the report are considered by many professionals 
as flawed; particularly in the areas of Science and English. 

• The comparison of college GPAs is extremely useful. 

• Use in designing remedial or developmental programs and targeting such programs. 

• Very little information listed 

• We need more data to compare multiple years to establish trends and better identify needs. 
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College Readiness and Success Section 

• Again, the trend data will be helpful. We are consistently working on raising our rigor for our lower levels. This 
reinforces the need to do so. 

• Considered our current use of placement into developmental and college level courses 

• GPA is a tricky measure. Students with high scores on math and science take more demanding classes so they may 
have lower GPAs. 

• I used this charts 6-8 mostly as a research model while writing amended Enrollment Management Plan 

• I would be interested in examples of how schools are using the various sections of the report. 

• Include Ncount in chart 6. 

• Increasing mandatory coursework in the 4 core areas that prepare students for college/career. This might mean 
ensuring there are rigorous courses in each core area beginning freshmen year that all students take and/or reducing 
the electives that can replace a core subject area. As many students as possible for example must be scheduled into 
Bio/Chem/Physics in grades 9-10 with increasing numbers of students being moved into that pathway each year is 
one example. The high school Math pathway includes identifying and moving prerequisite courses for high school 
into the middle school (such as Algebra I per CCSS alignment). 

• It is obvious that the students that are ‘college ready’ are going to have success at the next level. 

• More years of data are needed to establish trends that indicate necessary changes. 

• Most specifically with helping students plan their four-year high school curriculum choices. 

• N/A 

• Not sure 

• The core vs non-core information is not entirely accurate since it is self reported by students. Knowing that a 
student with a lower ACT score required remediation is not very helpful since many of the development programs 
are tied to ACT; the findings are not revealing. 

• Too much emphasis is given to the role of standardized tests. 

• Use in developing remedial and developmental programs. 

• What types of developmental coursework is needed by students specifically? Are there certain skills that are lacking? 
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College Success and Persistence Section 

• Board level reporting of successes and challenges 

• Determine the cause for the students not remaining in college after the 1st year. For example cost of the school, 
lack of academic progress, social issues with the school, etc. 

• Good as is. 

• GPA is not the only factor How do you show others? 

• I am not sure how much persistence is correlated with HS GPA or ACT. There are other variables that are more 
important. ($, attitudes, etc.) 

• I don’t know what a persister and non-persister is. There is no definition on the page 

• If we could drill in by unique ID to access the individual names of students we could follow up with some focused 
conversations. 

• Including persistence rates by ACT score band would be a useful addition to this section. 

• It is difficult to analyze persistence based on academic indicators only. 

• It is pretty obvious that in general, students with lower GPA’s do not persist at the same rate as students with higher 
GPAs - it might be more interesting to see the non-persisters broken out by various GPA ranges... are we losing 
high performing students? However, that is only helpful if we can also know whether the student dropped out or 
transferred to a university. 

• It is very useful as-is. The rate of college persistence is a huge concern for us, and we continue to try to increase it. 

• It would be nice to know that if they did not persist, if they were enrolled in another institution. 

• No suggestions. 

• Not sure 

• The implication of these charts is that students did not remain at the school for a negative reason. This may not 
be the case. For instance, they may have left a community college to attend a four-year institution. This section 
needs to contain more specifics and details (perhaps surveying students upon exiting from a college). Otherwise, 
I would recommend excluding this section. 

• The trend data will be most helpful... perhaps comparing us to “similar” schools across the state would also be 

helpful. We really have very little baseline to interpret if these values are “good” or “bad.” It is valuable. 

• This data in a vacuum with such a small sample size for districts our size is not the easiest to make any decisions 
from . . . 

• We are trying to track students who are identified as non-completers. 

• Which schools have higher averages of non-persisters? Are our students remaining in 4-year universities at greater 
rates than community colleges, etc.? 

• Why are students not persisting in the college/university that they initially enrolled? Need to include data from 
out of state as well as non-public universities otherwise the data we receive is VERY MEANINGLESS. 
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Appendix: Detailed Summary Information by Campus 

• “Agree”, taking into consideration the factors and suggestions provided earlier in this survey. 

• Comparing data between students enrolling in four-year versus two-year institutions, including core course 
enrollment, ACT scores, college GPA’s, etc. 

• Continue providing this information as it’s very helpful in making projections and predictions about future students 
with similar data scores. This helps us determine different ways to boost the scores of students who are at risk. 

• Department level discussions 

• Frequently paired with campus data for decision making. 

• Include a section of columns for Illinois enrolled students not enrolled at my institution. This would allow for an 
easy comparison between Illinois enrolled students at my institution and at the competitor institutions 

• Include information regardless of N size. 

• It does not do a good job of telling us which schools do a better job preparing students because top schools do not 
send large numbers of students to us. 

• It should be useful, but it tells schools little that they already did not know or infer 

• It would be helpful to have it labeled private and public schools, and 2- versus 4-year institutions. 

• Limited data available on our report 

• No suggestions. 

• Same as before. Patterns will be valuable. I would also love to compare to similar schools. 

• Table 1 (page 13) is useful, Table 2 (page 14) is useful only because it adds the developmental column (the rest is 
not needed), Tables 3-5 can be omitted, Table 6 is trivial (yes, student performance at a school increases with ACT 
score, but this provides little additional usable information), small n’s render table 7 largely useless, 20 is useful 
only because it provides persister data. 

• The current funding situation of the State makes this data MEANINGLESS. To compare students in my district 
who are receiving below the State average in per pupil funding to students in districts where the per pupil funding 
is 2 or 3 times the State average is not comparing apples to apples. 

• The exclusion of data for HS which sent us less than 10 students does not compute. I don’t believe it guarantees 
privacy while it does preclude faster schools which have small populations, i.e., George Washington HS 140842. 
The data, at the individual level is available anyway through other means. 

• Too many of my schools had insufficient enrollments for this section to be too useful. Once HS courses are aligned 
to the core competencies, it will be more useful. 

• Too much information for high school use 

• We use these appendices in conversations with our individual high schools. 
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Illinois Custom Addendum Charts and Tables 

• Add a cumulative % of total as N decreases so that we can quickly see where the 80% has been reached. 

• Any comparison to similar schools would be interesting. 

• Both formats are good ways to view the data in different ways. It is helpful to have both formats. 

• Disaggregated data would make the data more useful. While our students tend to do slightly better on the ACT 
than their peers also requiring developmental coursework, we really do not know the specific areas of deficiency. 

• Frequently paired with campus data for decision making. 

• Happy with them as they are. 

• I don’t need ACT data to tell me students need remediation work. I rely on the expertise of the classroom teachers 
that see the students everyday to tell me this information. Weighing a hog everyday is not going to make the hog 
grow as the old saying goes. 

• I’m confident in utilizing the charts as presented. 

• N/A 

• N/A 

• N/A 

• No suggestions 

• Not really sure what the addendum charts with student ACT scores having to take remedial courses is supposed to 
tell me — I find myself generally proficient in understanding data - but this gives me no information. 

• Since the figures are so close to the statewide averages, identify those that are statistically significant. 

• The charts I have do not have much data included. If they were more populated with data, they would be more useful. 

• There are too many schools listed. Focus on the 20 largest schools in each category. 
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Contact the IERC toll-free at 1-866-799-IERC (4372) 
or by entail at ierc@siue.edu. 

http: / ywww.siue.edu /ierc 


The Illinois Education Research Council at Southern Illinois University 
Edwardsville was established in 2000 to provide Illinois with education 
research to support Illinois P-20 education policy making and program 
development. The IERC undertakes independent research and policy 
analysis, often in collaboration with other researchers, that informs 
and strengthens Illinois’ commitment to providing a seamless system 
of educational opportunities for its citizens. Through publications, 
presentations, participation on committees, and a research symposium, 
the IERC brings objective and reliable evidence to the work of state 
policymakers and practitioners. 


